You should now see the LED array with the battery inside.

If you removed both caps you should be able to slide it out.

If you left the other cap attached, try popping that cap out and removing the attached battery with it (I didn't try this, but it should work).