In a heavy saucepan, melt chocolate over low heat. Add milk; cook and stir until smooth. Stir in vanilla. Making 1 s'more at a time, spread 1 tablespoon chocolate mixture over each of 2 graham cracker halves.
Place 8 or 9 marshmallows on 1 cracker; gently press the other cracker on top. Repeat. Store in an airtight container at room temperature.