Preheat oven to 350°. In a large cast-iron or other ovenproof skillet, melt 2 tablespoons butter over medium heat. Add apples and 1/4 cup sugar. Cook and stir until apples are golden brown and soft, about 6 minutes. Stir in cinnamon. Remove from heat. Stir in brandy; cook over medium heat until liquid is syrupy, 1-2 minutes. Remove from heat.
In a large bowl, whisk milk, eggs, vanilla and remaining 3/4 cup sugar until blended. Gently stir in bread; let stand until bread is softened, about 5 minutes. Stir in apple mixture and walnuts.
Add 1 tablespoon remaining butter to skillet; place in oven to heat skillet. Carefully remove hot skillet from oven once butter has melted.
Add bread mixture. Melt remaining 2 tablespoon butter; drizzle over top. Bake until puffed, golden and a knife inserted near the center comes out clean, 50-55 minutes. Serve warm.