Hi All.
I upgraded my kernel today to 4.14.5-1-vfio. I use the linux-vfio kernel from the AUR as my CPU requires the ACS patch in order to get workable IOMMU groups.
After upgrading I am now unable to start my Windows VM via virt-manger. I pass through SSD as a block device, which has been working flawlessly. When trying to start the VM now I get the following error…
Error starting domain: Cannot access storage file '/dev/disk/by-id/ata-WDC_WDS500G1B0A-00H9H0_164407800514-part4' (as uid:1000, gid:78): No such file or directory
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 125, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 82, in newfn
ret = fn(self, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/domain.py", line 1505, in startup
self._backend.create()
File "/usr/lib/python2.7/site-packages/libvirt.py", line 1069, in create
if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: Cannot access storage file '/dev/disk/by-id/ata-WDC_WDS500G1B0A-00H9H0_164407800514-part4' (as uid:1000, gid:78): No such file or directory
The path still exists.
[liam@RADAGAST ~]$ ls -la /dev/disk/by-id/
total 0
drwxr-xr-x 2 root root 700 Dec 17 19:23 .
drwxr-xr-x 8 root root 160 Dec 17 18:32 ..
lrwxrwxrwx 1 root root 9 Dec 17 18:32 ata-ATAPI_iHAS124_C_3524435_2C8230502346 -> ../../sr0
lrwxrwxrwx 1 root root 9 Dec 17 18:32 ata-KINGSTON_SH103S3120G_50026B722902764F -> ../../sdc
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SH103S3120G_50026B722902764F-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SH103S3120G_50026B722902764F-part2 -> ../../sdc2
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SH103S3120G_50026B722902764F-part3 -> ../../sdc3
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SH103S3120G_50026B722902764F-part4 -> ../../sdc4
lrwxrwxrwx 1 root root 9 Dec 17 18:32 ata-KINGSTON_SHFS37A120G_50026B7762033F87 -> ../../sda
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SHFS37A120G_50026B7762033F87-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-KINGSTON_SHFS37A120G_50026B7762033F87-part2 -> ../../sda2
lrwxrwxrwx 1 root root 9 Dec 17 18:32 ata-WDC_WD10EZRX-00A8LB0_WD-WMC1U6822885 -> ../../sdb
lrwxrwxrwx 1 root root 10 Dec 17 18:32 ata-WDC_WD10EZRX-00A8LB0_WD-WMC1U6822885-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 9 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514 -> ../../sdd
lrwxrwxrwx 1 root root 10 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514-part2 -> ../../sdd2
lrwxrwxrwx 1 root root 10 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514-part3 -> ../../sdd3
lrwxrwxrwx 1 root root 10 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514-part4 -> ../../sdd4
lrwxrwxrwx 1 root root 10 Dec 17 19:23 ata-WDC_WDS500G1B0A-00H9H0_164407800514-part5 -> ../../sdd5
lrwxrwxrwx 1 root root 9 Dec 17 18:32 wwn-0x50014ee65821a5c3 -> ../../sdb
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50014ee65821a5c3-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 9 Dec 17 19:23 wwn-0x5001b448b491f057 -> ../../sdd
lrwxrwxrwx 1 root root 10 Dec 17 19:23 wwn-0x5001b448b491f057-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Dec 17 19:23 wwn-0x5001b448b491f057-part2 -> ../../sdd2
lrwxrwxrwx 1 root root 10 Dec 17 19:23 wwn-0x5001b448b491f057-part3 -> ../../sdd3
lrwxrwxrwx 1 root root 10 Dec 17 19:23 wwn-0x5001b448b491f057-part4 -> ../../sdd4
lrwxrwxrwx 1 root root 10 Dec 17 19:23 wwn-0x5001b448b491f057-part5 -> ../../sdd5
lrwxrwxrwx 1 root root 9 Dec 17 18:32 wwn-0x50026b722902764f -> ../../sdc
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b722902764f-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b722902764f-part2 -> ../../sdc2
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b722902764f-part3 -> ../../sdc3
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b722902764f-part4 -> ../../sdc4
lrwxrwxrwx 1 root root 9 Dec 17 18:32 wwn-0x50026b7762033f87 -> ../../sda
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b7762033f87-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 17 18:32 wwn-0x50026b7762033f87-part2 -> ../../sda2
I have tried running virt-manager as root and running qemu as UID=0 and GID=0 with the same result.
Is anyone else experiencing the same problem?
Any help would be greatly appreciated.
tjw
December 18, 2017, 3:04am
#2
I’m curious about the -part4 bit. Are you sure you mean to pass through just a partition? Normally you pass an entire disk. It’s not going to know what to do with a partition without a table to describe it. I’d remove the -part4 from it.
Thanks for yourt reply.
Yeah, you’re right. That is somthing I had changed in my troubleshooting proccess. I was passing through the whole disk, and have changed the config back now.
I have tried reverting back to the LTS kernel, but the problem persists. Which leads me to beleive it is not a problem with the kernel.
I have attached my full VM configuration below.
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>win10-Test</name>
<uuid>39f86d2f-54e2-4f6d-802e-b001a623d71c</uuid>
<memory unit='KiB'>8392704</memory>
<currentMemory unit='KiB'>8392704</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement='static'>4</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='4'/>
<vcpupin vcpu='1' cpuset='5'/>
<vcpupin vcpu='2' cpuset='6'/>
<vcpupin vcpu='3' cpuset='7'/>
</cputune>
<os>
<type arch='x86_64' machine='pc-i440fx-2.9'>hvm</type>
<loader readonly='yes' type='pflash'>/usr/share/ovmf/x64/ovmf_x64.bin</loader>
<nvram>/var/lib/libvirt/qemu/nvram/win10-Test_VARS.fd</nvram>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<hyperv>
<relaxed state='on'/>
<vapic state='on'/>
<spinlocks state='on' retries='8191'/>
</hyperv>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough' check='none'>
<topology sockets='1' cores='4' threads='1'/>
</cpu>
<clock offset='localtime'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/sbin/qemu-system-x86_64</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source dev='/dev/disk/by-id/ata-WDC_WDS500G1B0A-00H9H0_164407800514'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</controller>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<interface type='network'>
<mac address='52:54:00:d6:ff:27'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x14' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
</hostdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</memballoon>
</devices>
<qemu:commandline>
<qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
<qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>
</qemu:commandline>
</domain>
tjw
December 18, 2017, 7:48am
#4
Concerning this part: -> ../../sdd
Does /dev/sdd exist? Do you use SELinux? What is the output of lsblk?
/dev/sdd definitely exists and is mountable in the host system.
[liam@RADAGAST ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 111.8G 0 disk
├─sda1 8:1 0 976.6M 0 part /boot/efi
└─sda2 8:2 0 110.9G 0 part /
sdb 8:16 0 931.5G 0 disk
└─sdb1 8:17 0 931.5G 0 part /run/media/liam/HDD
sdc 8:32 0 111.8G 0 disk
├─sdc1 8:33 0 450M 0 part
├─sdc2 8:34 0 100M 0 part
├─sdc3 8:35 0 16M 0 part
└─sdc4 8:36 0 111.2G 0 part
sdd 8:48 0 465.8G 0 disk
├─sdd1 8:49 0 450M 0 part
├─sdd2 8:50 0 100M 0 part
├─sdd3 8:51 0 16M 0 part
├─sdd4 8:52 0 464.8G 0 part
└─sdd5 8:53 0 461M 0 part
sr0 11:0 1 1024M 0 rom
I don’t use SELinux on this system.
I tried giving /dev/sdd 777 permissions temporarily but still get the same error.
tjw
December 18, 2017, 8:09am
#6
And if you change /domain/devices/disk/source@dev to /dev/sdd directly does it start working? QEmu shouldn’t have a fit with symlinks but this issue is a damn strange one.
tjw
December 18, 2017, 8:19am
#8
Sorry, that was XPath. Edit the domain’s XML and change
<source dev='/dev/disk/by-id/ata-WDC_WDS500G1B0A-00H9H0_164407800514'/>
to read
<source dev='/dev/sdd'/>
Edit: Also make sure nothing is mounted or in use on /dev/sdd.
Ahhh I see.
I tried that when I set the permission to 777. Just tried it again though and still no avail.
I’m going to work now, so wont have access to my PC. This is going to bug me all day
I might try restoring all packages to a previous date, though I feel this could end up causing more damage.
Just as I was about to blame Arch for being an unstable distro. I realise the problem was me all along
I have recently passed through a PCI sound card, which was PCI device 06:00.0 I wasn’t happy with the PCI slot the card was in; it was difficult to plug my headphones in.
I moved the sound card into a different slot and added it back into the VM as PCI device 04:00.0 via virt-manager. All the while assuming that 06:00.0 had been removed as it no longer existed.
The problem was PCI device 06:00.0 was remapped to the SATA controller for the drive I passthrough. Which explains the error Cannot access storage file '/dev/disk/by-id/ata-WDC_WDS500G1B0A-00H9H0_164407800514-part4' (as uid:1000, gid:78): No such file or directory as the whole controller was being passed through to the guest and the host could no longer find the disk by-id.
TL;DR I was passing through the SATA controller and the SSD by mistake. The kernel was not the problem at all, I am an idiot.
Thanks @tjw for your help
on to Looking Glass!
1 Like
tjw
December 19, 2017, 9:50am
#11
Glad to err… help haha, at least it was something simple. I am glad you got it sorted though.
(sorry Arch)