Preheat the broiler. Place 4 oven-safe cafe au lait bowls on a baking sheet.
Combine the milk and cream in a medium saucepan and bring to a simmer over low heat. Add the chocolate and whisk until smooth. Stir in the cayenne and vanilla. Divide the hot chocolate among the bowls. Top with 3 marshmallows each and place under the broiler. Broil until the marshmallows are golden brown and bubbling, 1 to 2 minutes.
Alternatively, line a baking sheet with parchment paper and spray with nonstick cooking spray. Spread the marshmallows on the paper and broil 1 to 2 minutes. Transfer the marshmallows to the hot chocolate. Serve immediately.