In a large bowl, dissolve gelatin in boiling water. Stir in cold water. Cover and refrigerate for 30 minutes or until syrupy.
Fold in 3 cups whipped topping; set aside. In a small bowl, beat cream cheese and sugar until smooth; gradually beat in milk. Fold in remaining whipped topping.
Grease the bottom of a 9-in. springform pan; sprinkle with cracker crumbs. Arrange ladyfingers around edge of pan. Set aside 1-1/2 cups of the orange mixture. Alternately spoon the cream cheese mixture and remaining orange mixture into pan. Spread reserved orange mixture over top. Refrigerate for 1 hour or until set.
Remove sides of pan. Garnish with mandarin oranges.