Hi guys! thanks for this awesome community, first time posting.
I’ve been in a hunt for the perfect server for my needs but I just don’t know what to go for, because my budget is limited. In an ideal situation, I think I would go for a Ryzen 3950x and 64Gb of ram for later upgrading to 128.
I want to run 2-4 Plex users, Bitwarden, less then 5 concurrent users on Nexcloud with OnlyOffice integrated, possibly a mail server like mail-in-the-box setup with the HAProxy-WI that @wendell did on Level1Linux and some VMs for testing linux distros and to create scenarios that I’m trying to implement at work so this way I can study them. Something like pfSense VM with a zentyal installation and 3-5 w10/linux vms to test GPOs and all the administration that can be done with it. I can’t go for a 3950x because of HDDs price, I was thinking in at least 8TBs x 4 in raid 10 on a XCP-NG hypervisor with Xen Orchestra.
Also, can I setup a fast ssd for cache with the raid 10 hdds?
Later on, can I just buy another HDD and add to the raid or do I have to keep adding 4 at the time for the raid 10 setup?
do they have to be the same or can they have different sizes and speeds?
Will a 3700x or a 3900x be enough for all of this?
what gpu will support this?
Can I have 2 GPUs passthrough? One for plex another one for a gaming W10 vm?
Wow, that’s a lot of tasks. Plex does not need a video card, so passing through one GPU to a windows VM is no problem.
I have a 3900x on my home server. I am running open media vault on ZFS. ZFS is a plug-in, Docker is also a plug-in. It runs plex and transmission, but qbittorrent is an option. I have not messed with VM’s on it yet.
My home server setup is here. Building a Home Server
I believe you may be able to go with just a Ryzen 7 3700X, at most a Ryzen 9 3900x IMO. I run 20 or more VMs on some 5th gen 8 core Xeons, so shouldn’t be a problem.
I’m not sure if you can do SSD caching on XCP-ng, but you definitely can on Proxmox using ZFS L2ARC. Nothing wrong with XCP-ng, I just seem particularly attached to Proxmox, so I’m biased (I’ve used XenServer before, but with Citrix XenCenter - it was quite enjoyable). From what I see online, it seems XCP-ng has some support for ZFS, but I never tried it, so can’t comment on it.
For GPU Passthrough, to avoid headaches, look for 2 different GPU models. I prefer avoiding nVidia because of their anti-consumer Code 43 when using VMs.
I suggest you use a separate, small boot SSD for the host OS (64 GB should suffice) and leave the HDDs and the caching SSD for the VMs. Keep in mind that ZFS will use some RAM, for 16 TB of usable space, around 16 GB (it actually uses less if the system needs that RAM and won’t be caching in RAM). So 64 GB of RAM should be a bare minimum if you intend to have 8 GB of RAM on at least 2 VMs (guessing Win10).
What do you think about a 2700x? Because it retails for about 188€, and the 3700x goes for 319€. that way in I can still upgrade for a 3950x down the line or a 4th gen if they’re compatible.
Tom Lawrence have showed that it is possible to use SSD caching with ZFS on XCP-ng I think, gotta revisit it
dthe thing with XCP-NG and Xen Orchestra is that I think it has a really great future ahead and it’s open source. Also, I want to implemented in work instead of having 2 srvs, so that way I can use it to study at home.
I was planning in using a 1060 6Gb that I had for plex and thinking in going for a 5700XT or the GPU that they launch on the upper tier.
with what could I connect to a w10 vm to play games? the monitor directly to the GPU, how to do it with in lan for multiple users?
Good enough. When doing multiple VMs, it won’t be running in full load anyway. Probably even a first gen Threadripper 8 core would be enough, however I don’t recommend 1st gen because of some small issues I heard about - they are great workstation CPUs though (also, not much of an upgrade path for TR4, since we now have TRX40).
Either will do fine. As I mentioned, I am highly biased towards Proxmox, because I used it more extensively and I know that you can easily do hardware passthrough in Debian, but I have no idea how XenServer / XCP-ng works other than usual VM stuff in XenCenter (also never used XenOrchestra, heard good things about it). That said, when in doubt and can’t find an answer, just try both for yourself and see what works. At work I got a Proxmox cluster, at home I just have Manjaro running KVM for my win10 VM.
Some VMs have trouble when running headless when remoting into them (ie: running 1024x768 resolution or lower), so definitely connect a monitor directly to the GPUs, even if it’s turned off. I have a TV with 3 HDMI inputs that I have my Linux host, my w10 VM and another w8.1 PC connected to.
You could play games via RDP, but I’d suggest you install TightVNC server on w10, connect to it via VNC, then use Steam and its in-home streaming feature for games. Not sure what you mean by multi-user though. Windows 10 is limited to 1 connected at once, either locally or through RDP. VNC functions like TeamViewer, but on the local network (albeit it’s terrible for fast moving content and has no sound). If you want to have multiple users connected at the same VM at once, you need Windows Server and additional RDC licenses for each user (not worth the cost). It’s cheaper to just make a 2nd Win10 VM and let another person RDP into that. Windows Server and multi RDP licenses only make sense when you got over 3+ users at once, when hardware resources become too expensive.
Technically you could illegally modify some registries to allow multiple RDC, but I don’t condone it and it will break Microsoft’s EULA.
I’m definitely going the 2700x route, but it makes sense do buy a good x570 MB?, for later upgrade. My questions now are more in terms of hardware rather than software.
Either way, I even might try them both and who knows what else, the more hands on experience the better, and once starting to have that opportunity, man oh man!
I was talking about RDP in for playing for ex COD MW and other smaller games but had heard about VNC a lot. But the possibility of for ex, making 2 W10 VMs for playing with my gf or brother seems very nice.
Wendell did some reviews about some B550 boards, he is pleased with them, however, IIRC most of them have problem with IOMMU groups (ie: they don’t separate the devices in different groups), which may be a good enough reason to go with x570. Look at Wendell’s reviews for more info, he covered quite a lot of them.
For RDP, I can’t help much. VNC doesn’t make separate session in Windows, with VNC you control the same screen. The advantage of VNC is that it’s just like controlling the PC locally and it’s really good for tech support, since you can have both local and remote control at the same time - but not different user sessions. For that, you need RDP with multi-session. And I never seen Windows Server gaming with multi-user session concomitantly. So I can’t really comment on it.
The reason why I’m running KVM instead of Proxmox is indeed because it has less overhead, because my PC has a Pentium G4560 (dual core with HT) and 8 GB of RAM. So I can only run 2 OS’es at the same time. I passed to the Windows VM a GT1030 (Manjaro uses the intel iGPU), a SATA port with a SSD (I can also dual-boot into Windows if I want more oomph) and a 1x PCI-E USB Card. I probably couldn’t run Proxmox, another Linux for my main and lame usage (web browsing, checking mail etc.) and a Windows 10 VM. The CPU is already struggling when I’m launching a small game like Asphalt 8 in the VM. It also doesn’t help that I have all my PCs in 2U chassis on a LackRack:
If I had a bigger budget, I would have gone with Proxmox, ZFS, ECC RAM and just separate each VM (doesn’t help that Manjaro has lots of hiccups for me lately). I would have also liked to host more services, like a separate FTP and Samba server, but eh, I can do just fine with storing things on my host OS and running rsync every night to sync the data to a RAID1 array with 1 TB disks (in the same PC). But I’m ok as things are now, I intend to buy a 4 or 8 GB Raspberry Pi, run it on an USB SSD and use that as my main PC and also build a separate low-power consumption PC (probably another ASRock J3455M like I have on my pfSense box, the white 2U chassis) to host my FTP, Samba and maybe NFS (thinking of making it just a NAS and running services on Pis).
