For the crumbs: Put the flour, brown sugar, granulated sugar, peanuts, cinnamon, nutmeg and salt in a mixing bowl and mix by hand. Pour in the butter and squeeze the crumbs tightly together, forming large or jumbo crumbs. Put the bowl in the refrigerator until the mixture is cold, 1 hour or overnight. (The crumbs can also be frozen for several weeks if wrapped airtight; defrost them for 15 to 20 minutes at room temperature before assembling the cake.)
For the cake: Heat the oven to 350 degrees F. Butter a 9x13-inch baking dish. Sift the flour, sugar, baking soda and salt together into the bowl of a stand mixer fitted with the paddle attachment. On low speed, mix in the butter 1 tablespoon at a time. Mix in the almond paste. Add the eggs and yolks one at a time. Mix in the vanilla bean paste. Add the buttermilk, then cream the batter on medium-high speed until very light and fluffy, 2 to 3 minutes. Fold in the chocolate chips by hand, then swirl in the peanut butter and cocoa nibs. Scrape the batter into the buttered baking dish. Smooth the top, tap the dish on the countertop and top evenly with the crumbs. Bake 20 minutes, then rotate the cake and bake until a cake tester inserted into the center of the cake comes out clean, 25 to 30 minutes longer. (After the first 20 minutes, I cover the pan with foil to keep the crumbs from getting too dark.) Remove the baking dish from the oven and set on a cooling rack until cool, about 25 minutes. Eat the cake for breakfast, warmed up for a snack, or serve with warm caramel sauce and ice cream as a dessert. The cake will keep for 2 to 3 days if stored in an airtight container at room temperature.