In a large bowl, cream butter and sugar until light and fluffy. Beat in egg. Combine the flour, nutmeg, baking powder and baking soda; add to creamed mixture alternately with buttermilk. Pour into a 9x5-in. loaf pan that has been coated with cooking spray.
Bake at 350° for 45-50 minutes or until a toothpick inserted in the center comes out clean. Cool in pan 10 minutes before removing to a wire rack to cool completely.