I’m not much of a VM guy but dabble when required. In the dark times before Linux there was virtualbox on windows, a few years later encountered Qemu in the context of some complicated setup emulating a console or something and dismissed it as beyond comprehension so stuck with VB. Recently had to dip into VM’s again and getting VB running on a recent kernel looks like one of those rabbit hole nightmares worth avoiding so tried Qemu again. For my light needs it just works, and installing it was just a matter of a repo package as it should be. Using virt-manager to manage VM’s via GUI, networking between host and VM is automatic. The most complicated I’ve gotten so far is a short ssh chain into the server and from there into a running VM, magic.
Have I been a dummy for not using Qemu/KVM all these years? Are there any gotchas on the horizon if I delve a little deeper? There’s some basic knowledge I’m still lacking like admin via CLI (virsh?), maybe live migration would be nice to know just for kicks. How about installing an iso distro through the CLI so it can be done remotely via ssh, is that trivial or do I have to figure out VNC to limp through a GUI install? Any other tips for things you consider essential for a mostly CLI use case please let me know, if it’s not stated in this post I’m probably unaware of it.
QEMU has been good for a long time, yeah. Even without KVM. I used to use it to emulate ARM, MIPS and PPC machines for testing software.
The downside has always been the huge number of command-line arguments necessary.
libvirtd helps a lot with that by saving machine definitions and building the command lines for you.
KVM can be independent of QEMU. QEMU is used for the virtual device support, mostly. Things like disk images and serial ports, etc.
One thing I am a little bit annoyed by is that things like KVM machine migration is limited to remote storage, and advanced features are often not really open source. I forget its name but Redhat has a virtualization package based on KVM and libvirtd with many of the same features as VMWare’s high end packages. However, you aren’t going to get that software unless you’re a Redhat subscriber.
For example, I wanted to suspend a VM on one of my servers, migrate its storage and then RAM to a second server. However I could not find any way to do that, it all seems to depend on having it based on an iSCSI block store. Bah.
I use a KVM/fileserver headless via ssh, moving away from using virtualbox a few years ago.
I liked the portability of virtualbox, for other host OS, but over time used this less and less. And (maybe, primarily) as much as I’ve liked Sun in the past I like Oracle in the inverse.
KVM/Qemu in these ways is not as straightforward to work with, at light use, as virtualbox but with virt-manager I have got it to do what I wanted.
Working only remotely, if you’re not already then I might suggest to export virt-manager to your local system display (if windows - I use VcXsrv to enable this) - you may then ssh -Y user@host (also configurable in PuTTY).
If you export virt-manager back this way, you can just view the VM local console from that.
You can also setup an iLOM-alike spice server to the VM, from within virt-manager. I sometimes connect to these directly via remmina, but with the above, it’s only really necessary if not wanting to ssh into the host (i.e. with your host local account) first.
you could also - install x2go on the host server, for a msrdp-like remote desktop.
Lots of kvm related things are still in development.
There’s things like vhostnet for efficient forwarding of network frames between VMs or between a VM and a physical interface without having to go through TAP.
There’s things like virtiofs for sharing a filesystem between VMs or between a VM and a host in a cache coherent way.
Getting a CLI install with virt-install is as simple as using --location instead of --cdrom then also passing --graphics=none --extra-args console=ttyS0. This has me far more excited than perhaps it should, but it’s the final basic piece that allows for full remote headless admin. Install with serial console, get ssh running on guest, acquire VM ip with virsh and job done
I’ve been avoiding VNC or any other way of sending graphics over the network because the LAN is glacial, it can just about handle the internet connection. It’s on the TODO to learn how to do remote graphics for things like graphical installers, Remmina looks good thank you.
I’m full Linux and FOSS at this point, the odd game in a blue moon excepted. So glad to be rid of Oracle. Thank you for all the suggestions.
I’m still saying the worst part about oVirt is that the documentation is desolate and there aren’t that many tutorials online. Other than that its pretty solid. Openshift on oVirt was a pretty seamless integration. Going to tackle the built in GlusterFS thingy next
I have also not done this yet (settled for iscsi fir now). If you figure it out, would be great to have a thread here on it (tag me if you remember to).
Once I get everything up and running I’ll probably do a mega dump about what I’m running, a la “What’s in your homelab” style. I’ll try to remember to tag you tho
You can do “full remote installation” using virt-manager quite easily by using the built-in SSH support:
On the “server”(VM-host) side all you need is libvirtd(e.g. apt install --no-install-recommends qemu-system-x86 qemu-kvm libvirt-daemon-system libvirt-clients netcat-openbsd), a user that is allowed to use libvirt VM’s(e.g. adduser username libvirt), and a SSH server(you probably already have one installed). The server is ready now.
On the “Client”, open virt-manager and select File->Add connection...,
tick the “Connect to remote host over SSH” box, and enter the username and hostname/IP of the “server” machine.
Ideally use a SSH key for login. virt-manager also ask for password or try to use SSH agents if I remember correctly…
You can now install/manage the machine as if it were a local libvirt VM through virt-manager, including the graphics output via spice or VNC(forwarded securely over SSH).
You can also still use virsh, e.g.: virsh -c qemu+ssh://username@server/system.
There are also web interfaces for libvirt, REST API providers, etc…
Did I mention that there are also LXC “drivers” for libvirt if you’re into containers(might need manual config tweaks).
You can also use the serial output (e.g. if you’re bandwidth limited) in virt-manager.
Regarding performance? Last time I checked the regular “compute” performance was basically the same across most hypervisors, and IO performance is great for Linux guests thanks to the virtIO stuff and KVM.
Very elegant, no proprietary software needed, also no GUI running on the server
IMHO a lot of people pay VMware and co. way too much money by simply not knowing this setup
I have between 5 and 7 M610 blades running the lab environment at work at any time*, each with 2x 512 GB SSDs.
Yes, this is a bit cringe.
One of those SSDs is set as a boot disk, and the other is used for the gluster.
It was a fair while ago that I deployed this, but there is a wizard during setup that allows you to specify which disks / volumes you want to use for the gluster.
I originally deployed it on 4x machines, I seem to remember it had around about a TB of available storage. I’ll see if I can find the notes next week.
The management VM runs on the gluster with its cute little crown icon, and it has been pretty dependable. I’ve done the usual testing like angrily yanking a blade out of the chassis to see what it will do, and it pretty much just gets on with it like you’d expect VMware to do.
I like oVirt.
Bit rough around the edges, but good.
That said, I also like xcp-ng.
*This is a slight exaggeration. We have just moved office, and I don’t even have a decent internet connection in the new building yet because of The Thing.
I’m looking at Qemu and Virtm now for setting up a basic Windows10 virtual machine image but its not very intuitive compared to VirtualBox DKMS solution. Any guides on using it for basic win10 vm? (this inc 3d driver setup if it has that).
What I would do to install a basic (Windows 10) VM:
(assuming you have downloaded a windows 10 ISO image and simply want the VM storage backed by a file)
File → New Virtual Machine
Local install media → Forward
Select ISO image, OS should be detected automatically. → Forward
Configure RAM/CPUs available(you can change this in detail later) → Forward
Enter a disk size for the virtual HDD → Forward
Enter a name, tick customize before installation, select a default network. → Finish
You can now view and change the VM configuration in more details, before the first boot begins.
Some recommendation: Under CPU check “Copy host CPU configuration” to make sure your VM uses all available CPU extensions.
The “max memory” allocation is typically the same as current. if you have the virtIO balloon driver installed you can use this to allocate more RAM at runtime.
For the first boot we’re always booting of the selected install media, so the boot options don’t apply just yet. Only after the first reboot.
You can change the “driver” for the storage to virtio(see below for notes).
Same goes for the network “driver”.
Some Linux guests can actually use 3D acceleration. To use it, change the listen type in the “display spice” menu to “none” and tick OpenGL. The under video select virtio and make sure to tick 3D acceleration.
Once you’re done in the configuration screen, click “Begin installation” in the top-left.
Install Windows 10 normally.
If you have setup virtio-based storage windows might ask you for the driver(or you need to specify them). You can change the current DVD image via the config gui: view → Details. You can change out the Windows 10 DVD image during setup with the driver DVD image no problem. In the HDD setup menu, click browse, navigate to the virtio DVD, and select the viostore for looking for drivers.
You can find the virtio drivers here
