Preheat the oven to 350 degrees F and lightly grease a 9 by 13-inch cake pan.
Sift the flour, sugar, baking soda and salt into a medium mixing bowl.
Put the butter in a small saucepan and, when melted, add the cocoa and water and bring to a boil. Add the boiling cocoa mixture to the dry ingredients and stir until just combined. Add the buttermilk, eggs, and vanilla and stir until smooth. Pour into the prepared baking dish and bake until a toothpick inserted in the center of the cake comes out clean, 35 to 40 minutes. Transfer to a wire cooling rack and sprinkle the marshmallows evenly over the hot cake.
Put the butter in a medium saucepan and, when melted, add the cocoa powder and buttermilk and bring to a boil. Remove from the heat and add the confectioners' sugar, vanilla, and salt and stir until very smooth. Pour the hot icing over the marshmallows on the hot cake and sprinkle with the nuts. Let cool to lukewarm before serving. Cake is best served warm or at room temperature.