Preheat oven to 350 degrees F. Line the bottom and over the sides of a 10-inch springform pan with parchment paper. In a cast iron skillet, over high heat, combine 1/2 cup of sugar, brown sugar, water and 2 tablespoons Armagnac. Season with salt and white pepper. Bring the mixture up to a boil and reduce by half, about 10 to 15 minutes, stirring occasionally. The end result will be a caramel like mixture. Remove the mixture from the heat. In a saute pan, melt the butter. Add the apples and saute for 5 minutes or until golden. Stir in the raisins and remaining Armagnac. Season the apples with salt. Spoon the apples into the lined springform pan. Pour the caramel over the apples. Place the pan in the refrigerator and chill for 30 minutes. In a mixing bowl, whisk the milk, eggs, remaining sugar, flour, and vanilla, together. Whisk until slightly smooth. (There will be lumps) Strain the batter over the apple and caramel mixture. Place the pan in the oven and bake for about 1 1/2 hours or until the cake is set and the center comes out clean with a knife.
Remove the cake from the oven and cool completely. After cooled, place in the refrigerator. Remove from the refrigerator and remove the parchment paper.
Serve the cake either cold or re-warm in the oven for a few minutes. Garnish each piece of cake with whipped cream, powdered sugar and fresh mint.