Need Help on New Build

Here is my build

To start off I installed all the hardware properly. When i do the first test to see if everything works my cpu fan spins for maybe 2 seconds and stops then nothing happens when i hit the power button again until i unplug the psu and plug it back in. So i wondered what caused this then i ended up removing piece by piece to see what the issue was. Finally, I removed my GPU from the PSU and the pci slot then my system worked. The light in front of the case was on, the fans were spinning on the cpu and the psu.

So my question is. Do i have to replace my 430W psu or replace my GFX card?

P.S I had no monitor or peripherals connected at this point.

If your computer has a speaker ( not the external one's), listen and see if it makes any sort of beeps when you attempt to turn on your computer with the graphics card in. If it does beep any amount of times, consult the motherboard manual. 

Try running your computer off your CPU's integrated graphics. If it works, go to device manager and try to uninstall the graphics drivers, turn the computer off, install the graphics card with the power, and if it boots, load the driver CD for the graphics card.