Okay so we like cheap we all do and we know for cheap we wont get much, but if your building your first desktop on an extreme budget and planning to use Linux then I have thrown together this little thing. http://uk.pcpartpicker.com/p/F2PXGX Its not powerful by any stretch, but if your programming, indie gaming, web browsing, word processing etc it will work wonders. I recommend a flavor of Debian also just because Debian 8 is due next week xD
Cheap and cheerful and the fact Debian is free or you can purchase a pre-made disk for couple of quid your good to go.
Raspberry Pi is a good SBC but its very limiting even with the RPi2, first problem is its based on Arm, while its a good architecture a new user would be a bit confused why they cant get Chrome on it. If your doing solid programming on a lower level RPi would do fine though :)
I agree the arm part does create some limitations, my first experience with Linux was with a raspberry pi. I liked it, I had a project with it, but school and stuff got in the way. Its a good cheap starting point, but if you are looking for something in the long run I think that build should do fine. I didn't look heavily into it either so I'm not 100% sure. Either way, have a good day... that rhyme was not intensional...
Meh, just get a Pentium or A4/A6. Not powerful by any stretch, but it works damned well. I'm surprised by the performance I get out of my G3220 build on IGP alone.