Prep the pan and oven. Preheat your oven to 350°F (175°C).
Grease a 9x13-inch baking dish with 1 tablespoon of butter.
Dry out the bread. If your bread isn’t stale, spread the cubes on a baking sheet and bake for 8–10 minutes until slightly crisp. This helps them soak up the custard without turning mushy.
Cook the apples. In a large skillet, melt 2 tablespoons of butter over medium heat. Add the diced apples, 1/4 cup brown sugar, 1 teaspoon cinnamon, and a pinch of salt.
Cook for 5–7 minutes, stirring, until the apples soften slightly but still hold shape. Remove from heat and stir in lemon zest if using.
Make the custard. In a large bowl, whisk the eggs, remaining 1/4 cup brown sugar, granulated sugar, remaining 1 teaspoon cinnamon, nutmeg, salt, and vanilla. Whisk in the milk and cream until smooth.
Combine. Add the bread cubes to the prepared baking dish.
Scatter the warm apples over the top, along with raisins and nuts if using. Pour the custard evenly over everything. Gently press the bread down so it starts to absorb the liquid.
Let it soak. Allow the mixture to sit for 15–20 minutes so the bread absorbs the custard.
If any corners look dry, spoon some custard from the bottom over the top.
Bake. Cover loosely with foil and bake for 25 minutes. Remove the foil and bake another 20–25 minutes, until the top is golden and the center is just set. A knife inserted near the middle should come out mostly clean with a few moist crumbs.
Rest and serve warm. Let it cool for 10 minutes.
Dust with powdered sugar and serve with a drizzle of caramel, warm maple syrup, or a scoop of ice cream.