Windows 10 Installer BSOD, Linux KVM

Hi,

I did previously have a working windows 10 install with my geforce 1070 being passed through and it worked. I’ve since bought a KVM switch so I decided to pass through a USB device too (one in its own IOMMU group) and it was also grand. Problem is it worked yesterday and today windows enters a recovery boot, so I decided to reinstall windows (because reinstalling windows generally helps right?) and the installer BSOD’s too. The BSOD stop code is: IRQL NOT LESS OR EQUAL (helpful as always)

When I force stop the VM and try to start it I get this:
Error starting domain: internal error: Unknown PCI header type ‘127’

Also another quirk is if I try to restart my host machine it gets as far as about to do the physical restart then hang indefinitely so I have to hold the power button until it powers off.

I’ve spent hours googling this and I’ve not yet found the answer, although I did find this: Error: internal error: Unknown PCI header type '127' but it didn’t help me with a solution

System Specs:

Ryzen 2700X
16GB Kingstone HyperX
Asus X470 Strix-f
Geforce 1070
FireGL V4900
Mint Linux 19.1

VM Config

<domain type='kvm'>
  <name>Windows10</name>
  <uuid>316aa8b8-7094-4234-b7be-a71b26d8f953</uuid>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/ovmf/OVMF.fd</loader>
    <nvram>/usr/share/ovmf/OVMF_CODE.fd</nvram>
    <boot dev='cdrom'/>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='nonvidia'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/kvm-spice</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/stuart/Downloads/Windows-10-06-02-2019.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='directsync' io='native'/>
      <source dev='/dev/sdb'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/stuart/Downloads/virtio-win-0.1.164.iso'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:1e:3b:85'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Content of /etc/modprobe.d/local.conf

alias pci:v000010DEd00001B81sv00001458sd00003723bc03sc00i00 vfio-pci #gfx
alias pci:v000010DEd000010F0sv00001458sd00003723bc04sc03i00 vfio-pci #gfx-audio
alias pci:v00001022d0000145Fsv00001043sd00008747bc0Csc03i30 vfio-pci #usb

options vfio-pci ids=10de:1b81,10de:10f0,1022:145f
options pci-stub ids=1022:145f
options vfio-pci disable_vga=1

All I wanted to do was play a little elite dangerous

That’s an error that’s typically related to the bus reset bug.

Which version of the windows 10 iso are you using? Some of them had issues with bsod in a VM.

I would try using q35 instead.

Additionally, it appears that your qemu is extremely old. Please update it with this PPA:

https://launchpad.net/~jacob/+archive/ubuntu/virtualisation

you shouldn’t be stubbing your device. That’s a deprecated method.

What guide did you follow? A lot of your configuration seems like it’s from knowledge that’s ~3 years out of date.

The windows ISO is the latest one 1809 i believe which is what I used initially when I had a working install

Yes I found that PPA on my googling the version of qemu currently installed is: 1:2.12+dfsg-0~18.04~ppa0 and thats from:
deb http://ppa.launchpad.net/jacob/virtualisation/ubuntu bionic main

I used these two of tutorials actually plus some extra stuff I found lying about. So 3 years out of date is about right.


Yes I the pci-stub that was left over from the last attempt so at least I knew I’d tried everything I’ve found so far, it was initially vfio-pci. Sorry I should have mentioned a bunch of this stuff up front, I wasn’t sure how much detail was needed.

Okay, let’s see what we can do to get you up to speed then!

I’ve had hit or miss success with that. Try switching to Q35. It might work.

Qemu 3.1, I think, is the latest stable. Let me see what I can do about finding you a newer package… :frowning:

Okay, so those alias lines, and the pci-stub don’t need to be there.

but you will want to add the following:

softdep nvidia pre: vfio vfio_pci
softdep nouveau pre: vfio vfio_pci
softdep amdgpu pre: vfio vfio_pci

Which basically tricks the kernel into thinking that vfio is a dependency of the normal GPU drivers, so it loads vfio first. That’ll make VFIO bind to the gpu properly. :slight_smile:

So it appears that qemu 2.12 is the latest available on Bionic due to package filters. I’m thinking you might be able to get away with just changing the sources.list line to the following:

deb http://ppa.launchpad.net/jacob/virtualisation/ubuntu cosmic main

The key difference being we changed bionic to cosmic.

That said, this may have unintended consequences. I’m fairly confident you’ll be okay though.

Well I’ll give that a go looking a little more at it this evening my other options are compiling 3.1 which i’ve done but its not working as yet a little more googling to fix it I think, hopefully that’ll work once i’ve fixed the kvm permissions problems