In a small saucepan, heat the 1 cup whole milk and 1 tablespoon vanilla extract over medium heat until it begins to simmer.
Remove from heat and add in the 4 bags Earl Grey tea, cover, and steep for 20 minutes.
Gently squeeze the tea bags to get the liquid out and pour the milk back into a measuring cup. Add additional milk if necessary to get the volume back to 1 cup. Let the milk cool to room temperature.
Once the milk has cooled, preheat oven the to 350°F. Line the bottom of three 6-inch pans with parchment paper and spray with cooking spray and set aside.
In a large bowl or stand mixer, cream together the 1 cup unsalted butter and 2 cups granulated sugar until light and fluffy.
Add 4 large eggs one at a time, mixing and scraping down the sides of the bowl between each addition.
In a large bowl, stir together the 3 cups all-purpose flour, 2½ teaspoons baking powder, 1 tablespoon loose Earl Grey tea leaves, and ½ teaspoon salt. Add to the butter mixture alternating with the tea milk, starting and ending with flour. Mix just until flour is combined.
Add 2 cups of batter to each prepared cake pan and bake for 30 to 35 minutes or until tops of cakes spring back when touched. Remove from oven and transfer to cooling rack to cool completely.