Hypervisor on Asus TUF X570 and some VMs... Where to plug what?

Hardware Hub Build a PC
1

Hi all, this is my first post, so please help me out if I put this in the wrong category.

I had an i7 4770 with a Nvidia 960 that that served me well since 2013 - Dual booting linux and windows. It is now plugged on the TV, and is used a gaming machine for the Kids, on a brand new smaller case.

What did I buy?

Ryzen 3600, 64Gb Ram, 500Gb Nvme, Asus RX5700, Asus TUF X570 plus motherboard. It is now running linux, and has 4x4Tb HD in Raid5, working as a cheap NAS for backup. My use for the card is mostly to work with photos on Darktable. I am restoring old family negatives as a hobby. I have very rare gaming moments but they do exist… :blush:

I have also plugged in a 3TB HD where my windows files live and two sata ssds (with my old windows and linux installs) from the old build.

What I’d like to do?

I was thinking to installing a Hypervisor on this machine - Proxmox - Linux directly - Unraid? I’d like to spam some VMs:

  • Plex Server (Windows)
  • File Server (linux ? directly from UNRAID? FreeNas?)
  • PiHole ( now running on a Pi)
  • Unifi controller
  • My linux main OS with my files and apps
  • A “work” VM (Windows) with some corporate software I need

I believe the machine can handle all this, but I am facing doubts on choices.

This motherboard has 8 SATA ports, 2 PCIex16 and 3 PCIex1 (one covered by the RX5700).

I also have available an old AMD Fury V4900 now resting in a shelf.

OPTION 1:

Run Hypervisor on Nvme; remove SSDs; use all SATA ports for HDs;

  • V4900 for the hypervisor, 1x16PCIe
  • RX5700 for Plex Server. 1x16PCIe
  • Realtek 4 port NIC for teaming [to be purchased] 1x1PCIe

OPTION 2:

Run Hypervisor on Nvme; remove SSDs; use onboard SATA ports for HDs;

  • GeForce GT 710 [to be purchased] for the Hypervisor
    . 1x1PCIe
  • RX5700 for Plex Server. 1x16PCIe
  • NT365T 4 port NIC. 1x16PCIe

OPTION 3;

Run Hypervisor on Nvme; keep SSDs for cache or fast access of data; use HBA Controller [To be purchase] for 5 HDs

  • GeForce GT 710 [to be purchased] for the Hypervisor
    . 1x1PCIe
  • RX5700 for Plex Server. 1x16PCIe
  • Realtek 4 port NIC for teaming [to be purchased] 1x1PCIe
  • HBA Controller; [to be purchased] 1x16 PCIe

The choice of Plex on Windows is due to the hardware acceleration and transcoding that lacks on their Linux server for AMD GPUs.

Those are the options I could think of.

If I had all this on my mind before going for this machine I would have got maybe another motherboard. Unfortunately not many choices available in the local market, and that was the only one with 8 sata ports. Just after I started playing around with it I understood the potential for consolidation of a lot of stuff I have in the house…

Please give your thoughts and suggestions. I’d like to hear also if you think it is not a good project for any reason…

Thanks!

2

Running two/three GPUs at once sounds a bit optimistic for a x70 chipset. You’ll want to research what the IOMMU groups ar for that board. I’m guessing some of those slots share groups with a bunch of other devices. In theory you don’t need any GPU for the hypervisor, it should be web managed, or via SSH.

PiHole is a great application for a Pi, why move it to your desktop?
Unifi controller is another great application for a Pi.
Plex server could run as a VM. If you have the Plex Pass and want to use GPU accelerated transcoding you’ll need to run that directly on the host OS, or pass through a GPU dedicated to the VM (see above concern about iommu). Quadro P400 is a good choice here, cheap and low power but still supports h.264 and HEVC encode/decode.

Your “work” VM probably doesn’t need a GPU, so you’re OK there.

The other thing to look at is how your USB controllers are assigned, can you pass an entire controller through to a guest OS, or do you have to assign individual usb devices?

SSD “caching” isn’t very exciting IMHO for unraid. It’s better under ZFS (FreeNas / Proxmox). Be aware unless you have a SSD designed for caching, it will experience excessive wear and a short lifetime. (Optane or used Facebook SSD would be my choices)

Grab a used Intel Pro/1000 card off ebay for network and use SR-IOV, or a simple bridge on your motherboard lan adapter, don’t need a quad port card.

3

Hi gordonthree,

You see, the main issue here is exactly what you described in the beginning of your reply - the IOMMU. I will take a look at it, but I think you are right from what I started seeing online.

Since Plex will not play nice on Linux + AMD I would need to run HyperV as the hypervisor. I’d like to avoid that. I see that a P400 is a good investment, but I’d like to use the RX5700 for this task if possible.

The PiHole and Unifi on VMs is just to have less hardware around.

Since I have already the NIC with 4 ports, I was planning to use this one and make bond.

I am not planning for Optane - the idea was to have the SDDs I already have to do the job. I was not at all aware that I’d need some part more specialized to avoid wear - good to know, thanks.

Concerning USB pass through… If instead of having Proxmox or Unraid I configure Linux as the main host, I could still install all the VMs and use the USB keyboard and mouse on the host, could I not? I cannot undertand exactly why I would need to pass USB to the VMs.

Thanks again!

4

This is news to me. I have the latest version of PMS running on my Threadripper server under Fedora 31. I’m planning to move it to a dedicated VM so I can better manage it’s resource usage.

Yes there’s not a problem assigning your keyboard and mouse to VMs, but it gets kinda old if you want to plug in a thumb drive, or external hard drive, etc.

5

Just to be clear - Plex works on Linux + Ryzen + AMD GPUs. But, under linux, it is not capable of hardware transcoding. It will transcode using the CPU only. That’s really unfortunate, and it seems that the issue is not being considered as a priority by Plex developers.

I even installed Emby here, but have not configured it yet, since it seems to handle transcoding from AMD GPU on linux out of the box. I will test it when I have time.

I will for sure try the IOMMU on this board soon, an I hope at least the Plex VM will work properly - and with acceleration.

6

Ah OK… I have Plex using a Quadro 400 that’s installed in an x1 mining riser.

Hey it can’t hurt to try one setup or another, that’s the fun of it right!?

7

Hi, I’ve found this thread:

I asked some more info - it may be that in the end I will be able to have this plex vm running… :slight_smile:

8

The TUF would be ok for a headless setup - I returned it due to pcie bandwidth issues & bought an Aorus X570 pro (in my setup I run 2 x m.2 + 2 x gpu + USB controller in pciex1)

9

Small update.

Just stump on my first roadblock.

I managed to pass through the RX5700 al right, but… the old FirePro will not work as main GPU due to some driver issues - and cannot handle UEFI. The open-source driver is buggy. The proprietary catalyst driver is not installable due to deprecated libraries from X.

Luckily I could ssh to the machine and revert the configuration removing the pass through of the new card.

I can run headless but defeats the purpose.

Tomorrow I will try to get an old card scraped somewhere and during the weekend I will try again.

10

You can check here which cards have UEFI BIOS

I’ve been using an MSI 4gb 560 Aero for over a year without any problems for the Linux host

11

I am about to buy this little toy: https://www.amazon.com/ZOTAC-GeForce-Profile-Graphic-ZT-71304-20L/dp/B01E9Z2D60 . It fits on x1 slot, and should be able to drive my linux easily enough.

This way I could get an LSI card and have a potential of 10 HDs in total.

On another x1 I could add another 2 port NW card, and bond the interfaces together so it act as a backup server for the house and a Plex streaming server.

12

So, here what I got:

8 SATA ports on x1 Pcie

2 Gigabit Ethernet on x1 Pcie

Video card Geoforce GT 710 x1 Pcie

Diskwise, what I have now:

  • NVME 512Gb with Arcolinux;
  • SSD 1Tb - right now has an old linux OS to be erased.
  • SSD 512Gb - Windows 10
  • 4x4T in Raid5 - My main data (backups of home laptops and other information)
  • 1x3T
  • 2x4T on an USB enclosure;
  • 1x3T in an USB enclosure;

What is the final plan:

  • NVME 512Gb with Arcolinux;
  • SSD 512Gb and SSD 1Tb ( to be decided)
  • 6x4T (Btrfs RAID 1 - I am afraid of RAID 5 mode).
  • 2x3T (Btrfs RAID 1)

Now, the main idea is Passthroug the AMD video card (RX5700) to a windows VM running directly from the SSD, perhaps. This will be my Plex server, and photo editing machine.

For now I am starting to backup and consolidate the data , so I can slowly transfer to the Btrfs by integrating the disks when they are free, avoiding the purchase of new disks.

I was planning to go with ZFS, and I may do it in the future, but I think that right now I get more flexibility with Btrfs, since I will not buy a bunch of 8TB disks. With the configuration above I will have 8 of 10 of 16 Sata ports full.

The remainder of material will arrive in a week or so, but the main time consuming task is moving the data around…

13

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.