Use a flathead screwdriver in between the caster and the base of the chair to pry the wheel out of the socket.

Turn the screwdriver gently until the caster starts to move upwards away from the chair.

Try rotating the screwdriver at different angles if the caster won't come out.