Is VFIO the correct solution for me?

Software & Operating Systems VFIO
, ,
1

I was planning on doing a new build for when Zen 3 come out and I been looking for months for the solution, is there is any, for the build I have in mind. I want a hypervisor and Virtual machines that I can load, pause and reload. I discarded many option and end with possible using Xen or KVM, I never used them I don’t have experience with them, just trying to learn about before commit. After I found that Xen will probably give me problem with drivers right now, I end here in this forum.

My Idea is to use a Ryzen 5950x (planning on buying when available) have a headless hypervisor, then 2 cores for a headless tiny Nextcloud server, 2 cores for a web-user Linux machine and 12 cores free to summon (load) what I need at moment of use, like a workstation, Linux gaming or windows-gaming machine.

I’m a system analyst. I do programing compiling, compression and decompression (that is where that 12 core will be use mostly), I play strategic and tactic games on my free time. What I have right now is a Laptop where I use for Web, email, discord, scheduling and so on, I have a workstation for work and I have a Nexcloud on Digital ocean that I rent online. My Idea is to upgrade all that in one machine, with multiple virtual machines.

I was thinking on using what I have on hand for this project and only buy the Ryzen 5950x because it will not get to waste if it does not works.

This is what I have:

Mobo: ASUS Strix B550-F
Mem: 32GB
Vcard1: RX 580 Nitro+ (may upgrade, I don’t play demanding games)
Vcard2: RX 550 (have it around, could use on the web-user VFIO)

So, Is VFIO the correct solution for me? can I do 2 VFIO and 2 headless? can someone point me to the right direction? or technology is not there yet?

English is not my first Language, sorry for any mistake and thanks for any response.

2

For home use go with KVM. It’s now as good as Xen, and has many life quality features that Xen doesn’t. Also has way better community support.
More things will work out of the box (like drivers).

Basically KVM with virt-manager is easy mode compared to Xen.
But hey if you want to learn more go with Xen :wink:

Not the best choice for passtrough, but since you have it already, then check it out and post IOMMU groups for others to profit :wink: (ls-iommu.sh on github)
But without passtrough it will be working fine. And I would put 2xNVME on it for VM images if it supports it.

Edit: Also i’m not really clear, what your plan is for display? That web-user VM? Why just not use host (HV) as your normal workstation and leave rx550 on it. Setup and troubleshooting will be way easier, since you’re new to this.

Or maybe you basing your plan on XenServer? If so, don’t. It’s not Xen/KVM.

1 Like
3

The plan is (KVM, now that I know is better for my case) :

Host: Command line Linux that auto load the web-user VM and the Nextcloud VM, so I can monitor the whole machine and logs. less point of failure, plus I can do a Btrfs on it for snapshots.

Web-user (VFIO1): Fedora (is what my laptop use right now), pass RX 550, monitor, KB and mouse.

Tiny server: Nextcloud

VM (VFIO2): pass the RX 580, 2nd monitor, 2nd KB and 2nd mouse. To a Linux for gaming, Linux workstation or windows gaming.

I want the ability to use a Terminal or browser, from the Web-user to control the host and load the VFIO2 with one of the 3 option mention above, turn off, load again, etc.

Not sure how possible is that, but what I’m reading it might.

I have a NVMe, a SSD and a 2T HDD. But planning on buying a 3 new 4T HDDs, make a raid 5, use the NVMe as a bcache and the SDD for metadata. My old 2T HDD is for backups, so I can keep all I have saved, while test this with the other storages. Plus second backup of the important data are on my laptop 1T HDD (Laptop have a NVMe too).

4

O, it is possible, no doubt about that, I use similar setups for production servers all the time. You just trying to use HV instalation as a pure “bios extension”. Only thing that can get in your way is IOMMU groups.

However, I use similar setup mainly on machines I want to access only remotely, and I still have IPMI/BMC on them, so they are not exactly “headless”

And since this supposed to be your home workstation I’m not sure, why you trying to make this hard for yourself, and create VFIO1 (your desktop) to manage HV, on which you are currently running your VM on, instead just using HV as your desktop, and control Nextcloud and Windows from it.

Because my prediction is, that you still be using your Laptop very often to ssh into your HV, to troubleshoot why your VIFIO1 didn’t boot :wink:

So my recommendation is, just install any flavour of Linux you like on your bare metal (HV) as desktop (rx550), and use virt-manager to create Nextcloud VM and Windows passtrough VM (rx580).

Use it for a while that way, get a lay of the land and learn a bit about KVM, because you will have roadblocks even with this simple setup.

And after you confident you “got this”, and you still want to make headless HV then just virtualize your current HV as VFIO1, thats it.

In terms of planning you may leave some small partition empty for this purpose, (partition you already planned as HV, but it can be even usb stick you add later)

Unless you want to show off, how haxxor you are, then well, good luck :smiley:

1 Like
5

I guess I did not notice how hard my idea is to implement. I was reading about IOMMU groups and B550 chipset, since you told me about the passthrough.

I just wanted to do a BTRFS of the total system pool of storage and use subvolumes instead of partitions. That way I can have segregate each machine so troubleshooting is more isolated and easy.

I was thinking that a headless bare metal HV that I don’t touch much is more stable and will not give me problems. But now I changing my plan to make it my desktop (rx550), but be extra careful and only do the most basics tasks and not bloat it.

Thanks: @Misiektw your are been very helpful. When I get the Processor, I will start posting the data for others to profit.

6

Don’t know about btrfs, never used it. But zfs-on-linux is solid and popular option. Just booting from it might not be supported on all distros.

You cannot really bloat Linux like Windows. Removing package doesn’t leave anything in system, except configs you made by hand. And there’s no registry to bloat.
My current install works like 2 years already, and it loads in few seconds like always did. Bios on x470 takes way longer to initialize :slight_smile: And I’m not particularly careful about software from repo/aur.

And if you have some Linux experience, for home use I recommend Manjaro with XFCE. But Fedora or Ubuntu are solid too. Just packages might be older. So, depends what you prefer in terms of new features/stability.

Sure, you’re welcome. Good luck with processor :slight_smile:

7

I started using btrfs since 3yrs ago and works fine to me. I tried ZFS 3yrs ago too, I know ZFS is better but could not make it work. Maybe I can give it a try now (putting on the list).

That is true, maybe is not bloat. But I do sometimes put too many things from Aur, using “yay” to my arch Linux, things get deprecated and the update is an ordeal of broken things and troubleshooting code I don’t fully understand. I been using arch on my workstation for the last 3yrs, I like I can use any program from any distro and useful for my work, but keeping things stable after a update, is like braking a vase and keep patching it with glue and tape, while liking. I like it don’t get me wrong, but no suitable for what I do on my laptop, I can’t have down times there. That is why I use btrfs, timeshift fix all errors, I can hit repeat and not game over. I know is my own fault, if I just use what is on pacman, use the LTS and do not deviate, all stay fine.

I been using the same fedora (home) since Red Hat 7.2. I just upgrade and keep passing to the next computer, and now is living on my laptop. But fedora require me to upgrade to each new version and even though is consider bleeding edge, I don’t mess with it. So is not suitable for what I do on my workstation.

I have a sdd with windows on my workstation that I boot when I play games I can’t on Linux. Just reboot, f12, select win boot disk and enjoy. I learned 12 years ago not to mess with dual boot. Boots on different disk is better, mix boots on one is a bad idea.

8

Yeah, pamac is default now on Manjaro and supports AUR. Sometimes acting weird too. But I found out that as long as I upgrade with “pacman -Syu” first, then upgrading just aur packages with pamac works fine.

Sure, if you know the tool already, i’m not gonna argue whats better (or butter in your case ;).

Yes, I recommend that too, for headache free experience. But I didn’t use dual boot since forever.

So I would probably just clone and use that as HV, but you probably want that “fresh new OS install” smell :wink: