I'd say, first try using a pair of tweezers to reach inside the headphone port and pull out the broken-off part.
If it can't be tugged out, I would consider opening the device following [guide|25074|this guide] until I could get to the headphone jack, and then trying to either push the broken part out if there's an opening in the back of the headphone jack, or to perhaps get the contacts inside the jack to push less firmly on the broken-off part to make pulling it out with tweasers more feasible.
If that didn't work, I'd probably resort to unsoldering and replacing the jack, but if you don't feel comfortable with soldering you might or might not want to try that.
I'm no expert; anyone else have a better idea?