Preheat your oven to 350°F (177°C). Line an 8-inch square pan (note: a 9-inch square pan is too large, see Notes for other pan size options) with lightly greased parchment paper or directly grease the pan. I typically use this square pan or this square pan.
Make the cinnamon crumb mixture: In a medium bowl, combine the brown sugar, flour, and cinnamon. Cut in the cold butter using a pastry cutter, two forks, or your hands (or you can use a food processor) until the mixture forms pea-sized crumbs. Some larger crumbs are fine. You should end up with just over 2 cups of crumb mixture. Refrigerate until step 5.
Make the cake: In a large bowl, whisk together the flour, baking powder, baking soda, and salt. Set aside. In a separate large bowl, use a handheld or stand mixer with a paddle attachment to beat the butter and sugar together on high speed until smooth and creamy, about 2 minutes. Scrape the sides and bottom of the bowl as needed. Add the eggs and vanilla, and beat on medium-high speed until fully combined. Beat in the sour cream, scraping the bowl again to ensure everything is well mixed. The mixture will appear a bit lumpy.
Add the dry flour mixture to the wet ingredients and beat on low speed until just combined. Then, add the milk and beat again. Be careful not to overmix. The batter will be thick, and you'll have about 2 and 1/2 cups of it.
Assemble the cake: Carefully spread about half of the batter into the prepared pan. Sprinkle 1 cup (about half) of the cinnamon crumb mixture evenly over the batter. Then, spread the remaining batter on top of the crumbs, using a spoon or small offset spatula to gently spread the thick batter. It doesn’t have to be perfect. Finally, sprinkle the remaining crumb mixture evenly on top.
Bake: Place the cake in the oven and bake for 35–40 minutes, or until the cake is fully baked. To test for doneness, insert a toothpick into the center. If it comes out clean, the cake is done. Allow the cake to cool in the pan on a wire rack for about 15 minutes.
Make the icing: Whisk together all the icing ingredients and drizzle it over the warm cake.
Serve: Lift the cake from the pan using the parchment paper edges and slice it into squares. If you didn’t use parchment, slice directly in the pan. Serve the cake warm or at room temperature.
Storage: Store leftovers tightly covered at room temperature for up to 3 days, or in the refrigerator for up to 1 week.