Prick the skin of peach and plum with a fork. Cut in half; remove pits. Cut each half into eight slices. Place in a greased 15x10x1-in. baking pan. Add the cherries.
In a small bowl, combine the butter, sugar, lime juice and rosemary. Spoon over fruit. Bake at 400° for 13-15 minutes or until fruit is tender, stirring occasionally. Serve over ice cream or frozen yogurt. Drizzle with pan juices.