Preheat the oven to 350°F (175°C) and grease the cake pan.

In a mixing bowl, cream together the softened butter and sugar until light and fluffy.

Add the eggs one at a time, mixing well after each addition.

In another bowl, combine the all-purpose flour, almond flour, baking powder, and crushed lavender.

Gradually add the dry ingredients to the wet mixture, alternating with buttermilk, and mix until just combined.

Stir in lemon zest, lemon juice, and vanilla extract.

Pour the batter into the prepared cake pan and smooth the top.

Bake for 30-35 minutes or until a toothpick inserted into the center comes out clean.

Allow the cake to cool in the pan for 10 minutes before transferring to a wire rack.

In a small bowl, whisk together powdered sugar, lemon juice, and milk until smooth.

Drizzle the glaze over the cooled cake.