Use a fingernail (or small coin) to push down on the battery door latch and remove the battery door.

The release latch is a plastic spring clip. If your Game Boy is getting on in years, be careful as the plastic may be brittle.

When you're replacing the cover, be sure to get its two plastic tabs into the slots in the bottom of the battery compartment before pushing the cover back in place.