Return to Level1Techs.com

3in1 - VFIO build

I currently have 2 separate PCs under my desk - server + gaming. I need new hardware and upgrading 2 PCs is just not optimal. I want to build a single machine and play with KVM and pci-pass-through. This way I should have free resources when I don’t game, and don’t have to deal with so much hardware and cables.

While I’m at it, I had enough of “routers”, I want my own, Unix-based one, so I’d like to cram it in.

Use-cases: single machine for 5 main roles:

  1. Server/NAS/host [host]
  • Linux based
  • Handle my 8 WD Red SATA drives (up to 10), mdadm
  • Docker
  • Qemu/KVM
  1. Sound Server [soundserver]
  • Isolated in Docker
  • PulseAudio, it takes multiple inputs from other PulseAudio over network, bluetooth, toslink-in, mixes and outputs to toslink-out.
  1. Multimedia PC [mpc]
  • Isolated in Docker
  • Emby multimedia server
  • Kodi
  • It will drive a 4K TV over HDMI 2.0
  1. Gaming PC [games]
  • Windows, isolated in Qemu
  • Has to drive my gaming monitor 1440p over DisplayPort
  • Has to drive my 4K TV over HDMI 2.1 in case I’d like to game on it
  • Parsec
  1. Router [router]
  • Pfsense/OPNsense isolated in Qemu
  • Handle double-WAN fail-over, 2x eth, 1Gbps down, 300Mbps up
  • OpenVPN server
  • Active IPS (Probably Suricata)

Here is the hardware (I had PC part picker as requested in the guidelines, but I can’t post any links):

  1. Asus ROG STRIX X570-F Gaming
  • mostly because of amazing IO
  • I’ll use all 3 PCIe x16. This will mean, both graphics cards will have PCIe 4.0 x8 instead of x16
  • I need WiFi for isolated IoT network
  • I need bluetooth for access to [soundserver] from mobile devices
  • 2.5Gbps eth will connect to WiFi access point (it also has 2.5Gbps)
  • 2.5Gpbs eth will be bridged with [router]
  1. AMD Ryzen 5800x
  • 6 CPU cores pinned to Windows for [games]
  • 2 CPU left free for [host] to use
  1. Cooler: Noctua nh d15 chromax-black
  • In general I want the build to be as quiet as possible, but I only don’t care much while gaming
  1. RAM 4x8GB, Crucial DDR4 Ballistix 3600 CL16 Black
  • In general, this is some kind of a nightmare, dual rank, single rank, different chips - my head hurts
  • From what I’ve gathered, Crucial 8GB are always single rank, but use 2 different chips. Worst case I’ll have to buy 3 sets of 2x8GB, and return one. Can’t figure better plan - I’m shocked, how complicated it is.
  • 16GB dedicated to [games]
  1. 2x SSD SAMSUNG 970 EVO PLUS 1TB
  • First pass-through to [games]
  • Second for [host], images (KVM/Docker), and other performance-sensitive stuff
  • yes, I know, they’ve changed the internals for worse in 2021, I don’t have any will left to deal with it after dealing with the memory mess :wink:
  1. Asus Strix Duo V2 OC GeForce 3070 (already have it)
  • Pass-through to [games]
  1. Gigabyte GF GT 1030, single-slot
  • not on Gigabyte website anymore, but available in shops.
  • HDMI 2.0b output, so [email protected] with DHR10+
  • [mpc] Emby live transcoding
  • [mpc] Kodi
  1. DeLock 4xSATA 3.0 PCI-e 3.0 x1 card
  • with motherboad, this will give me 12 SATA ports available on the [host]
  • I need to support up to 10x HDDs and 1x blueray.
  1. NIC: PCIe 2.0 x1 Intel I350 based, dual port
  • There are like 100x different manufactures of this card, I’m looking for a good deal.
  • For 2x WAN connections
  • I really need Intel, I’ve confirmed it plays great with FreeBSD and Suricata.
  • I can’t find anything PCI-e 3.0, but 2.0 should be fine anyway.
  • Pass-through to [router]
  1. AUREON 7.1 USB sound card
  • Actually I only need a single toslink-in, can’t find anything cheaper and Linux supported
  • for [soundserver]
  1. Case: I have an old, rare, mid-tower Corsair with 6x3.5" and 4x5.25" bays - can’t find anything even close, so it stays.

  2. Power supply: Corsair RM 750W (4-5 years old) - calculators say, this is all I need.

Things I still worry about:

  • Is 750W PSU really enough?
  • I’ve seen GPU benchmarks: PCIe v4 x16 changes nothing in compare to PCIe v3 x16. So PCIe v4 x8 should be totally fine, or am I missing something?
  • If 2nd GPU will have PCIe v3, won’t it force all CPU lines run in PCIe v3?
  • Will DirectStorage work? This is why I gamble to run Windows 100% in pass-through m.2, instead of installing OS on Qcow2 with all it’s benefits, and only install games on pass-through drive.

PS. State of the chip shortage in Poland:

  • I already have my 3070, found a good deal, only double the MSRP :wink:
  • there are problems with availability of SATA PCIe cards - Chia mining.
  • everything else is easily available for MSRP
  • there is no “scalping”, at least in traditional sens. Legal-tax-paying companies just sell GPUs for 2-3x the price, but they give a proper invoice and warranty.

PS2. I had links to all the parts, but forum forced me to remove them.