Intel 12th Gen CPU not working with QEMU Hyper-V nested virtualization

Hi all,

When booting with Hyper-V + host-passthrough it gets stuck at tianocore, does not change until I reboot which then loops into windows diagnostics which leads nowhere. Done using Windows 10, tried using newest windows version and 1909.

Specs: Manjaro Gnome 5.15 LTS, i5-12600k, z690 gigabyte aorus elite ddr4, rtx 3070ti.

I’ve spent days trying to figure out what was messing with it and it turned out I could boot when messing with my CPU topology, for some reason my 12th gen + Hyper-V + host-passthrough only works with sockets. Cores and threads above 1 causes boot problems, apart from disabling vme which boots, but the hypervisor does not load.

This fails (normal host-passthrough):

  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
  </cpu>

This boots (-can only change sockets):

  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="12" dies="1" cores="1" threads="1"/>
  </cpu>

This boots (-no hypervisor):

<cpu mode="host-passthrough" check="partial" migratable="off">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
    <feature policy="disable" name="vme"/>
  </cpu>

No matter what adjustment I do I cannot change the cores or threads or it will result in a boot failure, host-model just does not work once I boot the machine the host model changes to cooperlake.

My current way of bypassing this is I’ve downloaded the QEMU source code, gone through cpu.c and modified the default skylake-client CPU model to match my CPU, then I added in most of my i5-12600k flags manually, this seems to work with a 35-45% performance drop in CPU and in ram. Without Hyper-V enabled and using the normal host-passthrough I get near bare metal performance.

Tried with multiple versions of QEMU, EDK2, and loads of kernel versions (to add to this my i5-12600k gen does not work on kernel version 5.13 and below) even went ahead to try Ubuntu and had the same problem, my other (i7-9700k) PC works fine with Hyper-V. Also disabled my E-cores through bios resulting in the same issue. CPU pinning the P-cores to the guest does not seem to help.

Hyper-V host-passthrough XML:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
  <name>win10-hyperv</name>
  <uuid>89527fb1-3238-4d9d-8d5f-88d72323a21b</uuid>
  <memory unit="KiB">8388608</memory>
  <currentMemory unit="KiB">8388608</currentMemory>
  <vcpu placement="static">12</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-5.2">hvm</type>
    <loader readonly="yes" type="pflash">/home/apex/OVMF_CODE.fd</loader>
    <nvram>/home/apex/OVMF_VARS.fd</nvram>
    <boot dev="hd"/>
    <bootmenu enable="no"/>
    <smbios mode="host"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hap state="on"/>
    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vpindex state="on"/>
      <runtime state="on"/>
      <synic state="on"/>
      <stimer state="on"/>
      <reset state="on"/>
      <vendor_id state="on" value="valent"/>
      <frequencies state="on"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
  </cpu>
  <clock offset="utc">
    <timer name="pit" tickpolicy="delay"/>
    <timer name="rtc" tickpolicy="catchup" track="guest"/>
    <timer name="hpet" present="no"/>
    <timer name="tsc" present="yes" mode="native"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/opt/apex/bin/qemu-system-x86_64</emulator>
    <disk type="block" device="disk">
      <driver name="qemu" type="raw" cache="none" io="native" discard="ignore"/>
      <source dev="/dev/sdb"/>
      <target dev="sdb" bus="sata"/>
      <serial>F138F881</serial>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x8"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x9"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0xa"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0xb"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0xc"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0xd"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0xe"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0xf"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <audio id="1" type="none"/>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x01" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </hostdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value="-smbios"/>
    <qemu:arg value="type=17,manufacturer=Samsung"/>
    <qemu:arg value="-smbios"/>
    <qemu:arg value="type=4,manufacturer=Intel(R) Corporation,version=Intel(R) Core(TM) i5-12600K CPU @ 3.70GHz"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="input-linux,id=kbd1,evdev=/dev/input/by-id/usb-Corsair_Corsair_Gaming_K95_RGB_PLATINUM_Keyboard_1601902CAF7898E05F119ED5F5001BC0-event-kbd,grab_all=on,repeat=on"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Corsair_Corsair_Gaming_K95_RGB_PLATINUM_Keyboard_1601902CAF7898E05F119ED5F5001BC0-event-mouse"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="input-linux,id=mouse2,evdev=/dev/input/by-id/usb-Razer_Razer_DeathAdder_V2-event-mouse"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="input-linux,id=kbd2,evdev=/dev/input/by-id/usb-Razer_Razer_DeathAdder_V2-if01-event-kbd,grab_all=on,repeat=on"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="input-linux,id=kbd3,evdev=/dev/input/by-id/usb-Razer_Razer_DeathAdder_V2-if02-event-kbd,grab_all=on,repeat=on"/>
  </qemu:commandline>
</domain>

@wendell hey

Many thanks, any help is greatly appreciated!

Try 5.18-rc7 kernel

Full release is supposed to be 22.05. or 29.05.

Supposed to improve intel Alder Lake behaviour by including Intel’s Thread Director.

Please let me know the results. I want to do a similar setup when my parts arrive.

Hey, I’ve tried 5.18 rc6 directly from my Manjaro kernel app with no success I’m downloading the stable rc7 version directly from kernel.org and compiling it now.

I’ll update you if I make any progress.

Update: Compiled and installed 5.18-rc7 kernel, tried without any Hyper-V enlightenments just a default VM, no success. Enabling enlightenments still does not help, only boots when cores and threads are at 1.

Should I post this issue on the QEMU Github?

I have the same problem and the only solution so far was to install windows from scratch. It is likely a driver problem on the windows side. Did you change the host CPU model? I went from amd to intel before this happened.

Running my old Windows 10 Pro with only 2 sockets (Windows limit) when needed.