I have bought a dell poweredge r730 and a AMD firepro s7150 graphics card which I was hoping to use for hosting hardware accelerated VMs(VDI). I have created VMs before with KVM and pcie passtrough but I am struggling with getting my MxGpu compatible graphics card to work.
I am by no means a Linux guru but more of a hobbyist so assume my knowledge with both Linux and KVM is minimal. I am listing every thing I have done so fare so sorry for the wall of text.
I have looked at the AMD driver page for KVM which points to a github repo for GIM. It states that GIM has only been tested for a couple of old OS and kernel versions
The tested host OS for GIM is Ubuntu16.04.2. All other hypervisor SW(KVM, XEN, QEMU, LIBVIRT) versions are aligned with default version of OS.
To be more specific then it lists Ubuntu 16.04.2 server wit kernel 4.4.0-75-generic. GIM also supply a kernel patch which is made for 4.4.0-75-generic.
I have therefor installed Ubuntu 16.04.2 which default comes with kernel version 4.4.0-173-generic. I ended up downloading kernel version 4.4.0-75-generic and setting it as my default kernel in grub in order to follow the recommendations from the GIM repo as close as possible. After doing this then I get a problem with my ethernet adapters and lshw -class network does not list them any more. I think thats because the ethernet drivers are not available in that kernel version.
So as I got problems with my ethernet adapters I changed back to use kernel 4.4.0-173-generic. I cloned the GIM repo and compiled the driver. which gives me a gim.ko file. When I run make install I get the following error:
make -C /lib/modules/4.4.0-173-generic/build M=/home/miivers/projects/drivers/MxGPU-Virtualization/drv modules_install
make[1]: Entering directory ‘/usr/src/linux-headers-4.4.0-173-generic’
INSTALL /home/miivers/projects/drivers/MxGPU-Virtualization/drv/gim.ko
At main.c:222:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
DEPMOD 4.4.0-173-generic
make[1]: Leaving directory ‘/usr/src/linux-headers-4.4.0-173-generic’
To me it seems like signing the kernel driver fails. I therefor installed sudo apt install shim-signed and ran sudo update-secureboot-policy --new-key. When i tried running make install again and got the same error. I found a guide for signing kernel modules which I followed. Along the way I got the error EFI variables are not supported on this system. After some googleing I found a page which say that older Ubuntu versions has issues signing kernel modules on non UEFI systems. I restarted and changed my boot options from bios to UEFI and disabled secure boot. After doing this I could no longer see my drive in the boot menu(have installed ubuntu on the redundant SD card module). So I reverted these changes. I then tired to add “module.sig_enforce=0” to the linux command line /etc/default/grub and run sudo update-grub and rebooting. I ran insmod gim.ko. lsmod then shows:
Module Size Used by
gim 1007616 0
So it is loaded but not in use.
This is the rest of lsmod output:>
lsmod | grep -i amd
amdkfd 131072 1
amd_iommu_v2 20480 1 amdkfd
amdgpu 991232 0
i2c_algo_bit 16384 1 amdgpu
ttm 98304 1 amdgpu
drm_kms_helper 155648 1 amdgpu
drm 364544 4 ttm,drm_kms_helper,amdgpu
And the lspci output:
lspci -k
05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tonga XT GL [FirePro S7150]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Tonga XT GL [FirePro S7150]
Kernel driver in use: amdgpu
Kernel modules: amdgpu
The following lines is added to /etc/modprobe.d/blacklist.conf
blacklist amdgpu
blacklist amdkfd
I though this should blacklist amdgpu?
Well I am not sure what I should do next. To me it seems amdgpu is loaded even though I blacklist. The GIM driver seems to be loaded(manually) but not in use. Looking for input for what I should do next.