Proxmox cannot complete boot

Hello everyone,

I am currently running Proxmox VE 7.4.

I virtualized Ubuntu Server LTS 22.04 and TrueNAS Scale as VMs and started my first ZFS pool with 4 HDDs. Everything was working fine until recently, when I decided to increase number of HDDs inside my case.

After adding 4 more disks to my pool via HBA which is LSI SAS 9300-8i, the boot load starts and gets stuck with this message:

mpt3sas_cm0: overriding NVDATA EEDPTagMode setting

The aforementioned LSI SAS 9300-8i is flashed to IT mode and both VMs are set to manually start after the boot.

I have tried removing all HDDs from HBA, but the error persists. Only when I remove the HBA from the PCI-E slot physically, Proxmox VE boots normally.

I have seen maybe 2 or 3 threads online with this problem, but from what I understood is that message was only shown as a mere warning during boot for those users.

Something maybe worth noting is that the problematic message can be somewhat bypassed if I remove the following lines from /etc/modules:

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

However, then the WebUI of Proxmox VE becomes unreachable…

I am pretty new to homelabing/Proxmox so any help would be greatly appreciated.

Thank you in advance for your help!

are you passing the card through to TrueNAS or are you passing the drives through?

ProxMox is UUID based, but still manages to be weirdly sensitive to drive changes at boot. it will probably take making some manual adjustments to fstab after figuring out how ProxMox made drive associations.

Based on this Osris page some Dell HBA cards have a setting value that prevents proper use.

This fix requires the use of lsiutil

The example given will likely not work exactly as provided as they aren’t the same card; neither is an explanation for the fix provided so please use caution.

I am passing through drives, one by one. However, the VM with TrueNAS does not autostart because I wanted to set everything up properly before doing that.

this is not a good way to virtualize a NAS in ProxMox. as i said, ProxMox is very sensative to drive re-ordering and by passing through drives individually you cause a re-order situation fairly regularly.

do you have any VM backups of your virtual machines?

this is going to sound like a hail mary, but if your configs are relatively standard this should honestly be like 30 minutes of real work.

  1. take all the NAS drives out of your server.
  2. reinstall Proxmox
  3. load VMs (from backup hopefully)
  4. passthrough the HBA to the VM that will be doing NAS duty.
  5. install drives, and import ZFS
  6. Profit!
4 Likes

This is the way, thank you!

QFT

And passthrough of individual drives isn’t really a passthrough and more like a “share”. Technical process is very different. And sub-optimal for both configuration, troubleshooting and performance.

PCIe passthrough of the entire controller is how you do things. I tested the alternatives and it just sucked. Performance was bad, no SMART data, no proper drive labels,etc. Pretty much what you can expect from virtual disk vs native drives.

And then Unixoid first come first serve drive naming…just don’t. Proxmox doesn’t need the controller, your storage server needs it.

2 Likes