Upgrade 6770m 512mb to 6790 2gb card?

The iMac seems to go left and right on how the GPU is mounted in the system. Some of them use MXM and some of them solder it to the board. The only way to know is to open it up and make sure. It's realistically going to be based on the generation and how thin or thick the iMac is. For iMacs with MXM you can upgrade the GPU, in theory. The catch is probably going to be you have to use a Apple EEPROM flashed MXM card, which is not going to be cheap or easy to find. If anything, you probably won't be able to get one without haggling for a while. At least in the Mac Pro it takes it but it doesn't work with the GPU at startup since you get a black screen. I imagine the iMac will reject it flat out and not work at all.

These cards are a mess for backwards compatibility. They are not backwards compatible more often then not. You can't use a MXM 3.0a card in a 3.0b system like a normal desktop. You might be able to use a 3.0a in a 3.0b system but you can't use a 3.0b in a 3.0a system. You need to know which generation of MXM the machine uses to avoid a problem like this. The machine appears to be 3.0b, but double check this.

You can find the Apple OEM GPU on eBay or other places like OWC and poteitally Welovemacs, if your iMac doesn't have a soldered GPU. Even if you could, the gain is so small it's a waste of time and money. You are better off keeping the old card rather then replacing it.

My primary concern is if Apple deploys a GPU whitelist in firmware to prevent or at least limit this upgrade path. It's going to be a EFI limitation. You can upgrade it if there's a EFI whitelist, but not beyond what the whitelist allows.