Did anyone try to run VMWare Workstation on top of KVM/Proxmox?

Hi I’m trying to run Linux VM with VMWare Workstation inside KVM based hypervisor.

My LInux VM config looks like this:

<domain type="kvm">
  <name>OpenSUSE_KVM_oldsuse</name>
  <uuid>a2ddfa58-6c52-4b98-91e5-d9193f960055</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://opensuse.org/opensuse/unknown"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">33554432</memory>
  <currentMemory unit="KiB">33554432</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size="1048576" unit="KiB"/>
    </hugepages>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement="static">12</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu="0" cpuset="12"/>
    <vcpupin vcpu="1" cpuset="36"/>
    <vcpupin vcpu="2" cpuset="13"/>
    <vcpupin vcpu="3" cpuset="37"/>
    <vcpupin vcpu="4" cpuset="14"/>
    <vcpupin vcpu="5" cpuset="38"/>
    <vcpupin vcpu="6" cpuset="15"/>
    <vcpupin vcpu="7" cpuset="39"/>
    <vcpupin vcpu="8" cpuset="16"/>
    <vcpupin vcpu="9" cpuset="40"/>
    <vcpupin vcpu="10" cpuset="17"/>
    <vcpupin vcpu="11" cpuset="41"/>
    <emulatorpin cpuset="18-21,42-45"/>
    <iothreadpin iothread="1" cpuset="22-23,46-47"/>
  </cputune>
  <os firmware="efi">
    <type arch="x86_64" machine="pc-i440fx-8.2">hvm</type>
    <firmware>
      <feature enabled="no" name="enrolled-keys"/>
      <feature enabled="no" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" type="pflash">/usr/share/edk2/x64/OVMF_CODE.fd</loader>
    <nvram template="/usr/share/edk2/x64/OVMF_VARS.fd">/var/lib/libvirt/qemu/nvram/OpenSUSE_KVM_oldsuse_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pmu state="off"/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="off">
    <topology sockets="1" dies="1" clusters="1" cores="6" threads="2"/>
    <feature policy="require" name="topoext"/>
    <feature policy="require" name="tsc-deadline"/>
  </cpu>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="raw"/>
      <source file="/dmnt/fcraid/FCRAIDStorage/VMs/OpenSUSE_KVM_oldsuse/efi.img"/>
      <target dev="sdb" bus="sata"/>
      <boot order="2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <disk type="file" device="disk">
      <driver name="qemu" type="raw" cache="writeback" io="io_uring"/>
      <source file="/dmnt/fcraid/FCRAIDStorage/VMs/OpenSUSE_KVM_oldsuse/sdi.img"/>
      <backingStore/>
      <target dev="sdc" bus="scsi"/>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="2"/>
    </disk>
    <controller type="usb" index="0" model="none"/>
    <controller type="pci" index="0" model="pci-root"/>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/>
    </controller>
    <controller type="scsi" index="0" model="virtio-scsi">
      <driver queues="48" iothread="1"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x0d" function="0x0"/>
    </controller>
    <controller type="ide" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <audio id="1" type="none"/>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x25" slot="0x00" function="0x3"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x09" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x42" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x42" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x42" slot="0x00" function="0x2"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x42" slot="0x00" function="0x3"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0xd4" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x0c" function="0x0"/>
    </hostdev>
    <memballoon model="none"/>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x0b" function="0x0"/>
    </rng>
  </devices>
</domain>

so I believe it should support nested hardware virtualization? However VMWare doesn’t seem to like it. When I run VM with default settings it cries about being run under unsupported hypervisor. When I enforce nested virtualization using

vmx.allowNested = TRUE

It uses 100% CPU on all cores (on that Linux VM) and guest inside VMWare barely functions. And when I say barely I mean it - it took like 1 hour to reach login screen in Windows guest and I couldn’t even log in. It’s completely unusable and I would rather say that it “launches” than it “works”.

Did anyone try to achieve similar setup with any success?

1 Like

Coincidentally i stumbled upon this the other day.
https://openterprise.it/2022/12/running-nested-vmware-esxi-8-0-host-under-kvm-hypervisor/

It is the only time i heard of someone actually getting nested ESXi working. I have seen a lot of people try, and fail, to nesr ESXi.

1 Like

I tried to set up my network simulation software using a nested VM with KVM and VMware Workstation Pro 17.2. @lapsio got farther than I did. For me, VMware refused to even start up the virtual machine. I have a few theories on why my nested virtual machine failed to load.

First, in theory, VMware Workstation Pro doesn’t support nested VMs. Second, it is too limiting, meaning that if you try to share more than half of your available cores, it complains about it. What I would suggest @lapsio try is Proxmox as his hypervisor. I haven’t tried Proxmox because I don’t have another computer I could install it on. I hope to within a year, too. Proxmox, what I am waiting on is my new AMD EYPC Virtualisation Server.

There was a bug in kernel that did that. I noticed it around december and it was already merged to mainline kernel in january. But proper kernel with patch for nested virtualization applied only landed recently for Arch Linux so that’s why I got back to this topic. Previously it was giving some weird bug “blah blah doimain 0 something something”

1 Like