In a large bowl, cream 1/2 cup butter and sugar until light and fluffy. Add eggs, one at a time, beating well after each addition.
In a heavy saucepan, stir candy bars and remaining butter over low heat until melted. Remove from the heat; stir in buttermilk.
Combine the flour, baking soda and salt; add alternately with buttermilk mixture to creamed mixture, beating well after each addition. Fold in nuts.
Pour into three greased and floured 9-in. round baking pans. Bake at 350° for 30-40 minutes or until a toothpick inserted in the center comes out clean. Cool for 10 minutes before removing from pans to wire racks to cool completely.
For frosting, lightly grease the sides of a large saucepan with 1 tablespoon of butter; set remaining butter aside. Combine sugar and milk; cook over medium heat, stirring occasionally, until mixture comes to a rolling boil. Boil until a candy thermometer reads 234° (soft-ball stage). Remove from the heat; stir in marshmallow creme, chips and remaining butter.
Transfer to a large bowl; cool to 110°. Beat on medium speed until smooth, about 5-7 minutes. Spread frosting between layers and over the top and sides of cake. Sprinkle with walnuts if desired.