Computer Has No Video And Auto Boots

Recently I have been noticing my dell optilex not show any video and auto turning on. I tried to shut it off but as soon as I hit the off button and it shuts down I let go and it automatically turns on again. I believe that there is a problem with the video card that comes onboard because the paste that is on it is all dried out and isn't working. What I want to know was what is really wrong and if adding thermal paste on a GPU is ok. This GPU has a heat sink without a fan. Its just like a square with a bunch of ridges.