Preheat the oven to 350°F. Preheat the oven to 350°F. Line baking sheets with parchment paper, you'll need two.
To make the dough: In the bowl of a stand mixer fitted with the paddle attachment, add the butter, powdered sugar, and vanilla extract. Mix on low until combined and you don’t see powdered sugar anymore, there will still be a few chunks of butter but that is okay because it’s very soft.
Add the flour and contents of the Earl Grey tea bags into the butter mixture and mix on low until combined and smooth. The dough should feel like play dough in your hands.
Using your hands, form the dough into 30 balls and place them on the prepared cookie sheets. Flatten each one slightly so they are a round disk about 1 ¾-inch in diameter.
Bake for 15 to 17 minutes until golden brown. Sprinkle the tops with cane sugar immediately after coming out of the oven. Allow to cool completely on the cookie sheets.
To make the filling: In the bowl of a stand mixer fitted with the paddle attachment, add the powdered sugar, butter, and milk and mix on low until combined. Speed mixer up to high for 1 minute or until light and fluffy. Put in a piping bag with no tip.
Turn over every other cookie so they are all paired. Pipe a dollop of filling onto each cookie and sandwich them together.
Store in a cool dry place, or in an airtight container for up to 7 days.