I am attempting to get a virtualized install of TrueNAS Scale working on proxmox with my LSI IT mode controller passed through but my Scale never sees the drives on the controller.
When I start the Scale VM I get the proxmox splash screen then two error lines stating `error terminal ‘serial’ not found repeated it goes on to the grub lselection screen and on the install screen I do not see any of the drives attached to the LSI card. If anyone has any ideas on what I can try to get this working I am all ears. Below is what I have currently done.
I followed the proxmox guid at https ://pve.proxmox.com/wiki/Pci_passthrough
current grub settings
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"
The ouput of
dmesg | grep -e DMAR -e IOMMU' is [ 1.337815] pci 0000:c0:00.2: AMD-Vi: IOMMU performance counters supported
[ 1.337835] pci 0000:80:00.2: AMD-Vi: IOMMU performance counters supported
[ 1.337853] pci 0000:40:00.2: AMD-Vi: IOMMU performance counters supported
[ 1.337872] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 1.341284] pci 0000:c0:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 1.341291] pci 0000:80:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 1.341295] pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 1.341299] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 1.342328] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 1.342351] perf/amd_iommu: Detected AMD IOMMU #1 (2 banks, 4 counters/bank).
[ 1.342359] perf/amd_iommu: Detected AMD IOMMU #2 (2 banks, 4 counters/bank).
[ 1.342365] perf/amd_iommu: Detected AMD IOMMU #3 (2 banks, 4 counters/bank).`
the contents of /etc/modules is
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
the output of
dmesg | grep 'remapping'
is
[ 1.341303] AMD-Vi: Interrupt remapping enabled
the output of
find /sys/kernel/iommu_groups/ -type l
looks normal ex.
/sys/kernel/iommu_groups/55/devices/0000:40:08.2 /sys/kernel/iommu_groups/17/devices/0000:80:05.0 /sys/kernel/iommu_groups/45/devices/0000:04:00.3 /sys/kernel/iommu_groups/35/devices/0000:00:07.0 /sys/kernel/iommu_groups/7/devices/0000:c0:08.0 /sys/kernel/iommu_groups/63/devices/0000:49:00.0 /sys/kernel/iommu_groups/25/devices/0000:82:00.2 /sys/kernel/iommu_groups/53/devices/0000:40:08.0
the vm settings are
balloon: 0
boot: order=scsi0;ide2;net0
cores: 8
cpu: host
hostpci0: 0000:c5:00.0,rombar=0
ide2: local:iso/TrueNAS-SCALE-22.12.0.iso,media=cdrom,size=1692208K
memory: 8192
meta: creation-qemu=7.1.0,ctime=1672026924
name: scale01
net0: virtio=8E:22:15:E1:99:6B,bridge=vmbr0
numa: 1
ostype: l26
scsi0: local-lvm:vm-100-disk-0,discard=on,iothread=1,size=32G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=8a8e34a3-a38f-47d4-a6ff-9050fa96598f
sockets: 1
vmgenid: 1ea90faa-d98c-4b6e-bf6b-6686c3fb9a82
I have tried it with both i440fx and q35 machine types
My setup is
Proxmox 7.3-3
Fractal Design Define 7XL
SuperMicro H12SSL-I
EPYC 7542+ 2.9 Ghz
8x Crucial 32G CT32G4RFD432A
1x LSI 9300-16i
1x Intel X520-DA2 10G network card