Running Proxmox 7.3-6, attempting to passthrough an LSI2008-based HBA (purchased here) through to a TrueNAS Scale VM. HBA has 2x 2TB Seagate Barracuda HDDs and 2x 1TB MX500 SSDs attatched. All show up fine under Proxmox:
HDDs show up fine in the TrueNAS VM, but the SSDs report as 512 bytes and thus I can’t make an array with them:
System/Setup Information:
Computer Specs: i7 6950X - 4x4GB 2800Mhz DDR4 - EVGA X99 Micro 2 (all VM-related features enabled) - Radeon WX 2100 - ServeRAID M1115 (the HBA) + aforementioned drives - 250GB 960 Evo (OS drive) - Silverstone SST-TX300 PSU
TrueNAS VM Hardware:
Following the Proxmox Wiki PCI Passthrough guide here, IOMMU is enabled. Result of dmesg | grep -e DMAR -e IOMMU:
[ 0.009877] ACPI: DMAR 0x0000000039F72DF0 0000DC (v01 ALASKA A M I 00000001 INTL 20091013)
[ 0.009898] ACPI: Reserving DMAR table memory at [mem 0x39f72df0-0x39f72ecb]
[ 0.041224] DMAR: IOMMU enabled
[ 0.122521] DMAR: Host address width 46
[ 0.122522] DMAR: DRHD base: 0x000000fbffd000 flags: 0x0
[ 0.122527] DMAR: dmar0: reg_base_addr fbffd000 ver 1:0 cap 8d2008c10ef0466 ecap f0205b
[ 0.122529] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.122532] DMAR: dmar1: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.122534] DMAR: RMRR base: 0x0000003d434000 end: 0x0000003d443fff
[ 0.122535] DMAR: ATSR flags: 0x0
[ 0.122537] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[ 0.122539] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 1
[ 0.122540] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 1
[ 0.122541] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.122542] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.122543] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 0.123148] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 8.631097] DMAR: [Firmware Bug]: RMRR entry for device 05:00.0 is broken - applying workaround
[ 8.631100] DMAR: No SATC found
[ 8.631101] DMAR: IOMMU feature sc_support inconsistent
[ 8.631103] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 8.631104] DMAR: dmar0: Using Queued invalidation
[ 8.631108] DMAR: dmar1: Using Queued invalidation
[ 8.641863] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 11.891399] AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.
Remapping is enabled as well (dmesg | grep 'remapping'):
[ 0.123148] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.123149] x2apic: IRQ remapping doesn't support X2APIC mode
Does mention it does not support x2apic mode, I’m not sure if that effects what I’m trying to do or no. Should I try the override it mentions in the first output?
I have tried making the TrueNAS VM under i440fx, Q35 UEFI, Q35 BIOS (where it is currently, the TrueNAS installer did not like UEFI). Tried passing through the HBA with “All Functions” and “PCI-Express” flags checked. Opened the PC up, removed my ODD so I could run the SSDs directly off the SATA power cable from the PSU (I was using a SATA power → 2x SATA power splitter for them before), added in the 2nd SAS breakout cable and put them on there in case sharing a cable with the HDDs was the problem. No change.
Since adding the SAS card, the UEFI on this board does get a little derpy and puts me into a seemingly hidden tab instead of the “boot” tab, but it seems to work completely fine regardless. Said menu also confirms that VT-d with ACS Control and Interrupt Remapping is enabled by default when flipping VT-d on under the normal CPU config menu:
Any help would be appreciated, I would like to keep Proxmox as the host OS as I like to fiddle with VMs and TrueNAS Scale seems rather meh for that. Let me know if I need to provide any more information.
