Virtual Desktop Server

Hey Everybody,
I am trying to get a server setup that would allow a low power computer like a NUC or older laptop to pull a Linux desktop from a server.
I think my best bet is going to be using VirtualGL and TurboVNC with PCI Pass though. What I am trying to decide is what Hypervisor to build from. As a test I used Windows Server 2012R2 and did get RemoteFX working on a Windows 8.1 desktop, but what I want is a nice linux desktop that I can connect to from any of the devices from my house, that has full access to the GPU. So I played with Proxmox a bit but could never get it to work quite right.

So the question is has anyone done something like this?
Or just what Hypervisor would you recommend in trying to get something like this setup.

Thanks

personally i liked xenserver as a hypervisor. however i never tried with GPU passthrough because my server didnt support GPUs...if i ever get around to building a new one i might throw a couple in for some fun projects