In a large bowl, cream butter and sugars until light and fluffy. Add eggs, one at a time, beating well after each addition. Stir in ginger and vanilla. Combine the flour, baking powder, baking soda and salt; add to creamed mixture alternately with milk. Beat just until combined. Stir in crystallized ginger.
Pour into a greased and floured 10-in. tube pan. Bake at 350° for 65-70 minutes or until a toothpick inserted in the center comes out clean. Cool for 10 minutes before removing from pan to a wire rack to cool completely.