Return to Level1Techs.com

The Pragmatic Neckbeard 1: Intro

linux
helpdesk

#1

The Pragmatic Neckbeard's guide to Linux, KVM, ZFS and everything else in the world of Stallman and GNU.

AKA: How to steal Wendell's beard and get away with it

This is the first in a series of guides I plan to release on how to make the most out of your (admittedly overpowered) computer in a pragmatic way with Linux. In this first installation I'm going to simply discuss the requirements (physical, mental, temporal) and what you will get out of this in the end. The next article should be out within a week of this coming out.


That's nice and all, but what the hell are we doing?

Well, I'm so glad you asked. Essentially I'm switching to Linux but keeping Windows on the computer at the same time. The fact of the matter is that there are certain windows-only programs that I cannot live without. (Mass Effect and Skyrim, I'm lookin at you!) This means I'd either have to shut down my desktop and boot into windows every time I want to play a game, only to shut down and reboot into Linux to check my email, or, I just run a windows VM.

"But Sarge, VM's can't play games!"

Oh, billy, you're so naive. That was true in 2004. Today, Virtual Machines can do anything a physical machine can, with very few exceptions. We accomplish this with a technique called GPU Passthrough. Essentially, we're going to tell the host machine to not use the GPU and give it to the Virtual Machine. With a bit of black magic and trickery, we can make this work.


I'm going to start this off by saying that I don't have a lot of money to spend on computers. This is the primary reason I'm not building some crazy powerful server capable of running the L1T forums on its own. Let's talk about the hardware I have at my disposal.

  • CPU: i7-6700k (4.2GHz on air)
  • GPU, Linux: R9 380
  • GPU, Windows: GTX 970 (I know this sounds backwards, but this is how it works best for me, also I wanted to show that it can be done)
  • RAM: 16GB DDR4-2400
  • HDD: 4x4TB RAIDZ1, 2x128GB SSD 1 cache/zil, 1 os (btrfs).
  • Displays: 2x acer x223w, 1x A409u

I've got what I'd consider an upper-mid-range rig. Not quite 5820k, but not your average 3770. This all fits wonderfully within a Corsair 800D. I fully intend to put this under water at some point and push it to 4.6GHz, but that's not what this installation is about.

There are a few things people need to understand before they embark on this journey. First, you're going to want hardware cherry-picked to handle this project. While it's not absolutely necessary, you're going to benefit very much, in the form of less headaches and better performance, from having compatible hardware. That said, I recommend at least 8 threads (4 cores), 16GB of ram and an AMD fury series GPU for passthrough.
This project is not for the faint of heart. Not that you can't do it, but you're going to need an understanding of Networking, Linux and Windows.

Now that the warnings are out of the way, I can talk about the process and what it's going to entail.

1. Manjaro Linux


I'm using Manjaro Linux for a few reasons. First, it's like Arch. Second, it's not Arch. I know that sounds odd, but the fact is that Arch is a great OS, but arch needs constant maintenence to stay stable. I chose to use Manjaro because most of that maintenence is not required since Manjaro has it's own release cycle and stays mostly in sync with Arch while maintaining a higher level of stability. Manjaro also has a graphical installer wizard, which takes most of the headache out of installing Arch.

I'm using the arch package linux-vfio for my kernel which has been specifically tailored for GPU passthrough which makes it the perfect kernel configuration for us. Along with passthrough-specific configuration, it also includes certain patches which will help with compatibility.

If you're following along, I'm going to make the assumption that you've got enough experience to install an OS yourself. Just choose the Manjaro install that you like best. I chose the i3 community spin.

2. ZFS


It's probably clear by now that we're going to need a robust storage backend to support both the Linux host and the Windows VM. The clear choice for this is ZFS. There are a few benefits of using ZFS over BTRFS or LVM/EXT4 for our filesystem of choice which I'll enumerate here.

ZFS provides the best software RAID features on the planet. ZFS has managed to solve the Write Hole problem that plagues traditional raid solutions. ZFS supports thin-provisioned filesystems by default, along with thin-provisioned virtual block devices or volumes. ZFS has built-in read and write caching features, which can balance the benefits of SSD's speed and HDD's capacity for an incredibly powerful workstation. If this all isn't enough, ZFS checksums all your data and can tell if the data has been corupt and if you're using parity, ZFS can silently repair checksum errors. Just to put the icing on the cake, ZFS is Copy-on-Write, which means it doesn't overwrite data when it writes. This means ZFS can create an instant snapshot of a volume, filesystem or an entire array that only takes up space equal to the amount of data that's been changed. This makes ZFS an incredibly robust back-end for both your VM and your files.

3. VFIO, IOMMU and PCIe. The unbeatable trio.


The magical trio that makes your GPU go in your QEMU. No, that's not an inuendo, unless you want it to be.

IOMMU stands for Input/Output Memory Management Unit. At a very basic level, the IOMMU maps the PCI Express bus to memory addresses so you can read and write data to them. This will allow you to send your video card into a Virtual Machine and trick the machine into thinking it has physical hardware.

VFIO is basically a UEFI firmware for Virtual Machine's that allows you to perform passthrough. Along with allowing UEFI GPU's to initialize, it also supports UEFI bootloaders. This is helpful if you're using a very modern operating system. (Windows 8 or 10)

4. KVM


KVM, or Kernel-based Virtual Machine, is Linux' Virtualization system of choice. It's robust, and thanks to companies like Red Hat, its performance is at the top of the list. KVM boasts near bare-metal performance, within 1% of that of a physical machine, so it's the perfect option for what we're trying to do.

We're going to be using the libvirt system with QEMU/KVM where QEMU is the middleware that will interact with the KVM subsystem. This is a perfect system for our use case because libvirt allows easier management through a gui and XML file.

5. Raspberry Pi.


You're probably thinking, Where the hell does a Raspberry Pi fit into all of this? Good question. I've had plans for a while to use a raspberry pi as a sort of Command and Control system for a System like we're going to build. The end goal is that you can use a Single Mouse and Keyboard for the system without using Synergy to sync it all together. Synergy, I've found, is difficult to use and requires windows to be Logged in. This method will allow you to press a button connected to the Pi's GPIO pins and have the Pi switch the device that the keyboard and Mouse are connected to. You can also configure the Pi to send IR signals to switch inputs on your monitor if you've got the right hardware. That's right. $35 KVM switch. Theoretically infinte displays, theoretically infinite hosts. That's the magic of Software.

The Raspberry Pi will also tie into a Computer Desk project I'm planning, so I figured it was only logical to use it with the Passthrough section of the project as well. More about that project in another blog post.

6. More to come


I'm not the most organized individual. I'm trying to write down my thoughts as they appear, but I'm still working it all out. For now, be happy with this teaser. Ideally, I'll have them out on a weekly basis, but I have no plans for an actual schedule. (I am just too busy to commit to a day) Keep an eye on this post, where I'll be posting more information on my guides and topics. Also, when they come out, I'll be adding links inline so that navigation is easy.

Conclusion


This was just a brief introduction to my next few articles. I hope to captivate you in the next article, where I discuss ZFS and our storage options. If you want to see more from me, I'll be making blog posts from my newish blog at kebrx.github.io Hexo is shit, moving to ghost: blog.kebrx.tech. Until next time, happy hacking!

P.S. By the end of this series, you'll have a neckbeard strong enough to make this Linux God jealous:


GTX 1060 for GPU pass through question
Need Help on Setting Up Hardware Passthrough on Ubuntu 3930K
Linux News #4
#2

My double chin is aquiver for a neck bear you offer, will be following with interest


#3

Seems that PCI passthrough is becoming really popular. It's great because that means the vfio modules and OVMF will get more attention. But hopefully we won't need this in the future as more games and software comes natively.


#4

It is, you're actually the one who prompted me to get back into it. Thanks to your guide, I reloaded my windows VM and had a go at it again. There were a few things that you didn't cover that I plan on discussing though.


#5

Well thanks. :D

I'm sure there will be as you are going to use Arch and I used Debian. Not to mention that I also bypassed a lot of topics for the sake of brevity. If I went into detail about what exactly I was doing and how I figured it out the video would have been two hours long. So is there going to be a full guide or just these overview topics?


#6

The way I'm planning this, I'm going to make a post going into detail for each thing I outlined here. I just wanted to get this out there so I can push myself to keep releasing stuff.

I'm planning to more or less cover everything I know related to KVM and QEMU. Your guide was a good "here's how to get it done" sort of guide. Mine is intended to come to completion over the course of the next few weeks and cover a broader range of topics while giving a similar level of detail.


#7

YES. Definitely following this. Thank you so much for spreading your knowledge. I'm currently dual booting Windows 10 and Ubuntu Gnome and it's a pain switching back and forth constantly. I've also been looking in to alternative file systems like f2fs and zfs. (I'm running f2fs on my oneplus one). Just a question though, is this all done with one PC? I'm living in my extended Chevy Express converted van with a solar panel rack and battery bank powering my PC. Would it work for my lifestyle? (I mean I'd obviously have to get a newer cpu and gpu, running an i5 4670k and 980)


#8

Yep, just one physical PC. You're putting Linux on the bare metal and Windows on a VM within Linux.

Unfortunately, your PC does not support intel's VT-D processor feature, which means you won't be able to pass the GPU through to a VM.

I think it could definitely work for your lifestyle, but just check the intel ark site to make sure that the CPU (and motherboard for that matter) support VT-D and IOMMU features. I'm not a hardware genius, so I'm not the best to ask for specific hardware recommendations, but my setup works wonders for this.

If you keep the 980 and get something like an rx480, you should be good. Your other option is to use the Intel IGPU, but that's not good for Linux gaming.


#9

Article is looking interesting so far. I'd be happy to offer knowledge from my experience doing PCI passthrough to anyone following this having trouble, although I know there are others here with a lot more experience than me.

What's this about Synergy needing Windows to be logged in? I currently use Synergy on my PCI passthrough build without any major issues to date. I have never had trouble using it prior to Windows logon. After logon, the mouse usually resets back to the Linux host "screen" and I have to move it back onto the Windows "screen" once logged in, but that's the worst I've encountered.

As for difficulty, I've found that it can be a tad finicky, but in general I've actually found it very easy to use. I don't have anything sensitive to input overhead though, so that could be part of it. It's served me well in my build, but that's just my experience.


#10

Ima so jazzed bout this..... I like everything about this.... I even like the RPi implementation. I would like to see an overview of limitations of such setup and possible mitigation for limitations so we can take this and expand if we like to.


#11

That's the thing with synergy, I've found it extremely finicky to get set up on Windows. decided that since I'm using a custom-built DisplayPort switch for my setup, I'll build a custom keyboard switch that will interface with the DP switch as well. If synergy works for you, awesome and I'd love to chat about it at some point, but until I find my way with it, I'm not going to recommend it in a guide.

@gigabit I'll write a limitations post as part of the conclusion to the series. Part of the problem with putting limitations in the beginning is that this is kinda bleeding edge tech here, so in a month when this series is half done, we may have a different situation with some limitations. That and I haven't actually finished part 2 of the guide yet, which is part of why I'm not releasing it as one doc.


#12

Fair enough. It's worked well enough so far for me, but on my next PCI passthrough build in a few months, I plan on just passing a USB controller through so I can hotswap USB devices while the VM is running.

And yeah, I don't blame you for not recommending it in your guide if you aren't' comfortable with it yourself. If anyone is interested in trying it though, I'm happy to offer what I know of it.


#13

I'm going to cover that in the KVM section, as an afterthought while we're prepping the GPU for passthrough.

I'm comfortable with it on Linux, but I never can get the service to install properly on Windows, so if I can't, how am I supposed to expect someone who's following my guide to. If you've got any tips for that, I'd love to hear em.


#14

That's really weird. I've used it flawlessly for almost 3 months now since I started using PCI passthrough. It connects before Windows even finishes booting sometimes. The only issue I've had was it not finding my host, and for that I just entered the IP of the network bridge manually and set the server to bind to that address. Never had an issues since, that's why I recommended it in my video.

Just run the installer. That's all I've done and make sure to let it install Bonjour too.


#15

Hmm, Maybe I'm just special. I'll give it a try when I get off work.


#16

Regarding ZFS, I was curious what you'd list as ZFS could do that BtrFS cannot.

I mean... Oracle, OpenSUSE and it's Enterprise version, and one other I forget have moved to BtrFS as their main File System. From my research, ZFS is better for Parity RAID (i.e. the write hole), but everything I find saying one is better than the other, aside from that, is anecdotal.

Both:
Are Copy-on-Write
Use checksums
Use read/write caching

I could be wrong, I'm just saying.

I did find this as well:

https://btrfs.wiki.kernel.org/index.php/RAID56

parity data is not checksummed

That seems like a glaring omission though...


#17

I'm just not comfortable with BTRFS on production systems in raid 5/6. I'm using it on the laptop I'm currently typing on in a raid1 config, but there are some systems that I'm going to lean towards the more tested ZFS for.

I have plans to talk about BTRFS as well, at some point, but for now, I'm going to stick to the layout I've planned in the OP.


#18

Fixed the tag for you. In future just click on the edit title. There you can add tags also.


#19

Ah, thanks. I'm not too familiar with all this new-fangled forum witchcraft.


#20

@Vitalius, it's here.