VFIO with a potato Z97

For anyone else out there who wants to try VFIO with a potato:
This is a success story for a few years old Z97 system.
Specs:
Core i7-4790K BX80646I74790K
ASRock Z97 Extreme6 LGA 1150 Intel Z97
32GB of G.Skill RAM F3-1600C9Q-32GSR
IBM ServeRaid M1050 with 3 SSDs in Raid 0 (Relax I keep everything on a NAS)
Gigabyte 1070 TI GPU
Debian 9 with KVM/QEMU
Windows 10 Pro VM

Disclaimers:
I am not a heavy gamer. I play minecraft and try to quit EvE monthly. Doom was the last new game I played and it was awesome.
I used to manage SCCM templates. Roll your own Secure Host Baseline etc.
This computer was used to build them in Virtual Box.
I’ve run esxi, Debian, and pfSense non-professionally for years and still consider myself a linux noob. I professionally try not to screw up redhat on a weekly basis. Stronger on networks. Cisco, juniper, fortinet, and other self respecting cli.

Pros:
Free
Didn’t spend money on Cyber Monday.
I have the new found joy gaming in a VM I can treat like cattle.

Cons
It’s old.
Annoying IOMMU Groups that force weird compromises. The only way VFIO passthrough will work is with GPU in PCIe 2 (first full x16) that gimps the next full slot (4) But moving the GPU to 4 makes the IOMMU impossible for passthorugh. I’m taking a speed hit. With the Raid Controller or NVE in PCIe #2 I get about 1.2 GBs per second. In basically any other slot I’m seeing 970MBs per second. And have to settle because VFIO passthrough is worth it. I bought an nvme to experiment (nothing like seeing a new $170 nvme perform at similar rates to yester-years $700 3x ssd raid 0 array) Similar speeds problems thanks to m.2 layout. So there does not seem to be a compromise for this board. GPU in slot 2 or groups suck. That’s workable. 970MBs is nothing to sneeze at.

VM Part
I install Windows 10 Pro under KVM/QEMU using virt-manager (raw disk image and virtio controller), patch and update, activate(actually tweak fist for vfio then activate), tweak a little more to kill some telemetry. Then disable last access file stamping, disable hibernate, and temp disable pagefile. Defrag (why not) and use SDelete to zero fill the free space(I’m still using raw disk img at this point). Then the second best thing about this is virt-sparsify.
virt-sparsify Win10raw.img --convert qcow2 /var/lib/libvirt/images/w10l1.qcow2 (layer 1 more later)

This makes a smaller footprint (on your linux host) pristine windows 10 qcow2. Mine takes up about 18 GB on reasonably fast storage. And the beautiful part is next. I’ve got ram for days so do a:

sudo mount -t tmpfs -o size=16g /ramdisk && sudo qemu-img create -f qcow2 /ramdisk/w10rd.qcow2 -b /var/lib/libvirt/images/w10l1.qcow2

That’s makes an additional copy-on-write layer for the pristine copy of windows 10. In ram. Think of it as layer 2. Now open virt-manager make another windows 10 box and let’s call it amnesiac same settings as first vm just point the hard drive image at the ramdisk. I run nvidia and suffer kvm detection ‘feature’ so I paste this over everything between features.That means all browsing or questionable links can’t touch the pristine copy of windows 10. Windows boots up in seconds because it’s always optimized. And any unoptimized crap is in a ram drive layer. If you’ve got the storage space, keep the raw img file so you can install updates when you want to refresh the baseline. I have a simple script that copies the ramdisk layer to and from permanent storage. That way I don’t have to waste bandwidth on multi downloads of MS updates. My Ramdisk cow layer is about 3GB. adds maybe 3 seconds to boot time to move back and forth from non volatile storage. There are more pros and cons of this setup. Adobe premiere would probably blow your ram cache in seconds. I’ve been installing games to disk images that can be treated as just another big file when not in play. I would like to try to install a game then update the prime windows layer to see if the game breaks. Kind of a poor man’s thinapp, but expirements necessary.

VFIO:
It works, and it’s the best part about this setup. Honestly specifics here are just not important. Setup is not much of a difference between Archwiki and Debian sources really. I suffer the nvida driver disable 'feature so google up how to add the following to your vm xml.

vendor_id state=‘on’ value=‘VirtualNoob’
hidden state=‘on’

I’m not running any new games on this setup yet. I have more fun doing this type of work than playing games usually. eve and minecraft are smooth but that’s not going to saturate 1.1x pcie bandwidth so I’ll need to look into the q35 chipset and passing the root bridge (non unique id… yuk) but that’s low priority as it works fantastic for what I use the machine for, learning. Occasional microsoft office kiosk. etc.

Gotchas:
Still a work in progress.
I have no idea what I’m doing, but it’s fun.
It’s been worth the time.
Do not install 10 VM in bios mode then activate. You will loose part of your soul calling microsoft for reactivation for ovmf uefi.

3 Likes

Awesome stuff! I have exactly the same motherboard and thought right of the bat, that it is too old to do VFIO. You proven me wrong. :slight_smile: On Windows I run Assetto Corsa in VR together with wheel/pedal/handbrake USB devices. So that makes it quite a complicated setup for a passthrough. But I’ll have to try, as now I know it is possible.

1 Like

There is one gotcha in that platform. 4770k doesn’t work has to be 4790k or non k. That problem is why I wound up building a new system last year

I am lucky twice. :smiley: My CPU is the same as OP’s as well.

2 Likes