VFIO AM4 - Need help finishing (most of) my build

Hardware Hub Build a PC
, ,
1

Hi there,

My current rig has been built up more than 10 years ago with minor modifications since then.
It’s time to move on, and I’m (hopefully) ready to finally make the leap to VFIO (if only to harden/isolate that awful W10…).

The Goal

Gaming on a Win VM, possibly also on a Linux one (see Caveats).
Either use VMs for applications isolation on the hosts and/or on a Linux VM (see Final Thoughts/Questions).

Available Parts

I don’t have much to go on for a new build, but the one part that this will have to evolve around is this one:
CPU AMD Ryzen 9 5900X 3.7 GHz 12-Core Processor

Other notable parts
GPU AMD ‎Radeon RX 6900 XT 16 GB
Case Corsair Obsidian Series 800D ATX Full Tower

Caveats

  • Money is rather scarce. That’s the main reason I’m keeping the 5900X, seeing how much more pricey parts are for an AM5 build. I might make exceptions if I deem them worth the extra expense, though…
  • Host distro: Since I often see it used, it’s probably a good idea to specify that I really don’t fancy Ubuntu (nor its derivatives, of course). I’ve seen guides for Arch and Fedora, but are there more ? Debian and the likes (I’d prefer Devuan, but day-dreaming won’t help me get this done) ?
  • Host GPU: I suppose just about any GPU would do ? I have a couple GTX 670s lying around (so one for backup, if need be)…
  • Region: Europe. Shops-wise, I’m mostly looking at CaseKing, possibly eBay and, as a very last resort, Amazon (feel free to share any respectable shops you’ve had good experience with).

Parts I Considered
Mobo ASRock X570 Taichi Razer Edition ~357€
Mobo ASRock X570 Taichi ~316€
Mobo ASRock X570s PG Riptide ~183€

I’ve sticked to ASRock, seeing as it’s often used in VFIO builds.
As I understand it, the main interest of the Razer Edition over the regular Taichi is mostly about the number of M.2 slots… though there’s also Thunderbolt 3 and 2.5Gb LAN, both which I can most certainly live without.
The Riptide is not only interesting because of its price, but also its lack of WiFi, which is a big plus for me (to clarify, lacking it is the plus).

Feel free to weigh in about those… or even others.
CPU Cooler Noctua NH-D15 Chromax Black ~120€

Black is the new black !
Thermal Compound Prolimatech PK-3 ~15€

I don’t really know what’s what these days when it comes to compounds, as I wrote this one down a couple years ago…

Parts I Have No Clue About

  • RAM: I was thinking 64GB, so I could experiment with multiple VMs and such, but I’ll probably start with 32GB, if only because of the cost (unless that’s not much of an issue with DDR4 ?). 2x16GB, then, but I don’t know what timings and speed to focus on… Since I have no plan to o/c, I take it 3200MHz is the only real option ?
    The thing I’m somewhat worried about is the table for the Riptide on ASRock’s website (as an example):
    4xMatisse-3200-2667
    Does that mean that 3200MHz won’t work with all slots populated (that would at least validate going for 2x32GB from the get-go) ?!
  • PSU: I’m partial to Seasonic and modular models, but once again, I’m definitily off the loop there…
  • Fans: Current ones are Scythes (6x140mm & 1x120mm), and some are noisy above 30% throttle… I suppose I could replace a bunch.
  • SSDs: NVMe… yet another black hole in my hardware-fu. From my understanding, I should be looking at PCIe 4.0 and Type 2280 (though that’s less of a problem).
    While speed is always good, I would possibly have it superseded by reliability (I like the idea of having cache to lessen the wear, for instance).
    Edit: I’m currently leaning strongly towards MLC with cache.

Final Thoughts/Questions

  • I suppose I can also have a Linux distro in a VM, with or without access to the main GPU (possibly with a toggle of sorts, if I choose to play on the Win VM while doing other stuff on the Linux one, and at some point shut down the Win one and game on Linux) ?
  • If I’m limited with the number of NVMes I can plug directly, can I use more with some adapter, or would I be better off with “regular” SSDs ?
  • On that subject, are NVMes that much faster than SSDs (I understand that having NVMes sitting right on the motherboard can give them some advantages compared to using SATA cables) ?
  • Somewhere in the future, I’d like to have a 10G LAN, and so I’ll need a free PCIe slot to that end.
2

I dug some more on a couple of the subjects listed in Parts I Have No Clue About, and while it provided some answers (well, hints, at the very least), it also brought some more questions :crazy_face:

RAM

Regarding the number of sticks to go for, it would appear that 2 is the sweet spot (it seems to mostly be an o/c thing, but not only), so 2x32GB it will be.

On the subject of frequency, it’s still fuzzy.
While most of the modules I’ve been looking at are listed as 3200MHz, it appears that this value can only be achieved when activating XMP in the BIOS.
That shouldn’t be an issue with the mobos I’ve listed above (and probably none at all actually, since I don’t plan on going for entry-level models, in case that’s where the limit is), and it shouldn’t be a problem with the VM either… probably ?
What I still don’t know is: can I pick (for instance) a 3600MHz kit and have it run at 3200MHz ?
I suppose if it’s listed as 3600MHz, it means that it won’t get worn out faster just because it gets more voltage ?
Also, in the few reviews I checked about those 3600MHz kits, the reviewers always also had the CPU o/c’d… is it mandatory/linked, or could I just “o/c” the RAM at the default XMP values and leave the CPU alone ?

When it comes to picking a brand/model, it gets even trickier… looking at QVL for those ASRock boards has been giving me headaches (of sorts).
If we keep it about 3200MHz (if only for the sake of argument and simplification) and 32GB modules:

I like to think that this “evolution” is merely some models disappearing from the shelves (it’s old stuff, after all), and that everything should be compatible between those 3 mobos, but I don’t know for sure.
What I do know is that all the 3 remaining modules vetted for the Riptide were just as well for both the Taichis.
As far as hints go, that’s big enough, but the real problem is being able to buy ye olde parts…

To add to the conundrum, the only one of those three 32GB modules that I could find as buyable in my usual stores is the F4-3200C16Q2-256GVK (Ripjaws V), which makes no sense because for those 3 QVLs it’s listed at 32GB with that exact reference, which is actually an 8x32GB kit, when the max RAM those mobos can take is 128GB… :exploding_head:
Anyway, I did find a review (from TweakTown, IIRC) of the 2x32GB kit and to summarize: “meh”.
Since it’s the only QVL-vetted I could find in “my” shops, I suppose I could use it as a fallback solution if I can’t find anything better.
But even then, there’s two variations listed: [email protected] vs [email protected]… the former being literally twice as expensive than the latter :japanese_ogre:

The one promising kit/model was BL32G32C16U4B.M16FB1 (Crucial Ballistix), but it seems to be gone for good…

SSDs

Until proven otherwise, I do quite like MLC and cache, but there’s not many models to be had besides Samsung’s 990 PRO.
Fun fact: On Samsung’s own website, the 990 PRO is listed as MZ-V9P2T0B/AM (MLC) for US, MZ-V9P2T0BW (MLC) for UK, and MZ-V9P2T0BW (TLC) for France…
Complicated seems to be the new simple :triumph:

Regarding the use of NVMe over “regular” SATA SSD, is there a point in having the Linux host on NVMe ?
From my (limited) understanding, once the VM is fired up, it has its own hardware, including NVMes
Depending on the mobo I go for, it could free up an M2 slot I could use for a Linux VM.

E.g.: with any of the Taichis (3 NVMes), I could probably go for:

  • Linux host: SATA SSD
  • Windows VM: 1TB NVMe (OS) + 2TB NVMe (Games)
  • Linux VM: 1TB Nvme (OS + games)

I imagine I could also go for a 4TB NVMe for the Windows VM, but wouldn’t having games and the OS on the same module wear it faster ?

In case that’s not obvious, I like the idea of stuff that last :nerd_face:
The 4 SSDs I have in my current rig have been there for more than a decade, with no issues (so far)…

Motherboard

  • Taichi
    • PROs
      • 5 PCIe slots (3 GPU slots: x16 or x8 / x8 or x8 / x8 / x4)
      • 3 M2 slots
    • CONs
      • If the 3rd M2 slot is used, the last PCIe slot will be disabled (not that big of a deal)
      • WiFi/BT
      • Expensive
  • Taichi Razer Edition
    • PROs
      • 4 PCIe slots (3 GPU slots: x16 or x8 / x8 or x8 / x8 / x4)
      • 3 M2 slots
      • 2.5Gbits LAN (can’t hurt)
    • CONs
      • If the 3rd M2 slot is used, the last PCIe slot will be disabled (limited expansion)
      • WiFi/BT
      • Expensiver
  • PG Riptide
    • PROs
      • 6 PCIe slots (GPU slots bifurcations unclear)
      • No WiFi/BT
      • 2.5GBits LAN
      • Cheapest
    • CONs
      • 2 M2 slots
      • Crappy audio chipset
      • If the 2nd M2 slot is used, a couple of onboard SATA3 plugs will be disabled (not a problem at all)

Besides price and WiFi (or the blessed lack thereof), the ultimate choice might be about NVMes and PCIe slots.

Since I’m hoping to build up a 10G network at some point in the future, I’ll need a free PCIe slot for that, and the Taichi Razer is the most limited on that front (when using 3 NVMes).

The PG Riptide is quite limited on the NVMe front, and I don’t know if it’s worth it (both money- and performance-wise) to get a PCIe card to have a third (or even fourth) NVMe slot(s).

Regarding GPU speeds (for lack of a better term), is there a way to have x16 on the first GPU and x4 (or even x1) on the second, instead of x8 / x8 ?
I suppose x8 / x8 and the likes is for SLI or Crossfire, or is it something that can’t be helped when using 2 GPU cards ?
If so, is it possible to have one GPU in the first GPU slot (x16, for the VM) and another in the third GPU slot (x4, x2, x1, whichever, for the host) ?