Ouch! I’m suspecting your dedicated GPU board is going on you.
Give this a try reboot your system and press and hold the Option (⌥) key is the screen stable then?
You might also try running diagnostics as well
Reference: Mac startup key combinations
We don’t have a direct GPU board guide but this will get you 90% there Installing iMac Intel 27" EMC 2429 Dual Drive Kit (HDD or SSD) follow it to Step 33, pop off the SATA & power cables for the HD & optical drive, as well as the boards power connector so you can fully remove the logic board, carefully ease it out.
Once the logic board is out flip it over to locate the GPU thermal sensor
Remove the screw holding the heat sink to the frame
Then remove the four screws on the board
Carefully lift the heat sink off
Your system came with either a MXM-A or MXM-B board
Remove the two last screws holding the board in and ease it out of the socket.
Here are your choices for replacement boards:
- AMD Radeon HD 6770 512 MB - P/N 661-5967
- AMD Radeon HD 6970 1 GB - P/N 661-5968
- AMD Radeon HD 6970 2 GB - P/N 661-5969
You might be able to recover the board by heating the board up to fix the cold solder joints that develop. You’ll need to apply some flux onto the board and using a heat gun heat up the board so the solder re-flows but don’t go to far! As you don’t want the components to shift. Some people just cook the board, I’ve found that technique is not as dependable as doing it properly. Here’s a time-lapse of someone using the heat gun approach YouTube - GPU Repair
You’ll also need to clean off the old thermal paste & thermal pads, and will need to put fresh thermal pads on the RAM chips (don’t use thermal paste).