Is it possible to remove all the desktops in my house and instead, use a big computer with a few VMs with hardware passthrough and access them remotely, like if they were normal computers? What I would need to build such a thing?
Yes. Its a common tactic in companies and schools where frequent re-imaging of computers might be required.
You need to setup what are know as "Thin Clients" (can be very basic machines) that auto reconnect through RDP upon boot.
Depending on how many computers you plan to virtualise and how heavily hit they are depends on the server's requirements. If you only expect, say 2 desktops to be used for light office work and web browsing, then a good Quad core with a decent amount of RAM would be fine. Disk systems are then a little more tricky as you have to accommodate file systems for however many machines whilst keeping them response, even under load. A nice RAID 5 on HDD's will give you good speed and capacity for n-1 the number of disks == to number of machines. (Eg 4 disks == good for 3 machines under continual read/writes). If capacity isn't and issue you could RAID 10 some SSDs.
If a machine needs a GPU for say OpenCL/CUDA work, then you'd need to take that into consideration. For Intel machines, you need a CPU and board that support VTX-d for hardware passthrough. Under AMD, pretty much any system allows for IOMMU.
Do not expect to game on this, unless you plan to attach the monitors used for output directly to the serving machine. As RDP is mainly used over a network there is additional latency which is easily noticed when a low response time is needed.
Talking about network limitations, multiple Gb NICs will give you the best performance. If you have 1 Gb NIC for each connecting device, I wouldn't bother with LACP, just get each connecting client to address each NIC via individual IPs. (Aka only have one machine bridged to each interface) If there are more connecting clients than NICs, then look at using LACP but make sure the switch supports that type of operation. Alternatively look at a 10Gb NIC as the main backbone. Much easier to setup, but alas a higher cost of entry for both the NIC and supporting switch.
Anyway TL:DR edition. Add all the accumulative power of all the machines you wish to virtualise and aim for something of equal power.
This was only basic, but if you want additional info just ask.
-zanginator
What kind of budget are you working with?
also, what kind of equipment you have at hand?
I'm going to lurk on this thread as i want to learn about this.
Pretty much all pc's are fast enough to run desktops vm's without issue - only biggest performance impact is limitation of hdd r/w; Since you want to replace your desktops i'm not going to draw out some pricy stuff.
I recommend getting VMWare esx to cut down on OS header, normal power friendly xeons type E3 (they should be normal price and use less power) and a lot of ram(just think of it 4GB+ per desktop if you want to have win7). The most important part of the build is hdd's. I recommend getting a raid solution with 4+ hdd's that can be connected with gig port or fibre (usb is not good) *Raid 1 *if you have will to sacrifice a little bit of your write speed for security - read speeds are still better because it reads from all drives* or Raid 0 more speed for less security (like 1 hdd breaks, and you'll loose data ;< ) You don't have to worry much about the actual capacity of those drives - as there are scripts on vmware esx that eliminate duplicates of files - resulting in more space becuase who needs system files and its folders 5x or so... when you simply can have 1.
Thank you all for the info, it's for a future project, but i still don't have any money... i was just looking if was possible to make such a thing, but i'm kinda sad about the latency issues... even with a 10Gb network this would be a problem?