Step 1: Preheat your oven to 375°F (190°C). Lightly grease a 9x13 inch baking dish.
Step 2: Arrange the brioche bread cubes evenly in the prepared baking dish.
Step 3: Prepare the cream cheese swirl: In a medium bowl, beat the softened cream cheese with 1/4 cup granulated sugar, egg yolk, and 2 tablespoons milk until smooth and creamy. Set aside.
Step 4: Prepare the pumpkin custard: In a large bowl, whisk together the 6 large eggs, pure pumpkin puree, sweetened condensed milk, 1 cup whole milk, pumpkin pie spice, vanilla extract, and salt until well combined and smooth.
Step 5: Pour the pumpkin custard evenly over the bread cubes in the baking dish, ensuring all bread is moistened. Dollop spoonfuls of the cream cheese mixture over the custard-soaked bread. Use a knife or skewer to gently swirl the cream cheese into the pumpkin custard.
Step 6: Prepare the pecan streusel: In a separate medium bowl, combine the flour, brown sugar, and chopped pecans. Cut in the cold butter cubes using your fingers or a pastry blender until the mixture resembles coarse crumbs. Sprinkle the streusel topping evenly over the entire casserole.
Step 7: At this point, you can cover and refrigerate overnight, or bake immediately. If refrigerating overnight, let it sit at room temperature for 20-30 minutes before baking.
Step 8: Bake for 45-55 minutes, or until the top is golden brown, the streusel is crisp, and the custard is set (a knife inserted into the center should come out clean).
Step 9: Remove from oven and let stand for 5-10 minutes before serving. Serve warm, with optional maple syrup or whipped cream.