Preheat the oven to 350 degrees F. Line cupcake tins with cupcake liners.
For the cupcake batter: With an electric hand mixer, cream together the butter and granulated sugar until light and fluffy. Add the milk, vanilla and egg and mix until incorporated. In a medium bowl, sift together the flour, baking powder, cinnamon, ginger, nutmeg and salt, and slowly incorporate into the wet ingredients on low speed until combined. Fold in the apple pieces.
Pour the batter into the cupcake liners (mini cupcakes are our favorite!) until about three-quarters full and bake until a toothpick or knife comes out clean, about 20 minutes for regular-sized cupcakes, or 12 to 14 minutes for minis. Set aside to cool.
For the frosting: While the cupcakes cool, use your hand mixer to combine the powdered sugar, cream cheese, cinnamon, vanilla extract and salt, adding a bit more sugar if you prefer sweeter frosting. Beat until smooth, and refrigerate until ready to use.
Before topping each cupcake with frosting, insert a squeeze bottle filled with the caramel into each cupcake, about halfway in, and fill with caramel. Frost the cupcakes, and then top each with a small slice of apple dipped in the caramel, a drizzle of caramel or a sprinkle of cinnamon--or all three if you want to be a badass.