I will refer to my mega chart, please hold after i math out some solutions for you…
relevant thread:
GREAT QUESTION
ZFS managed HBA backplanes don’t care where you are plugged in, RAID controllers DO care what port you’re plugged into as ZFS/ReFS uses UUID for drive identification.
You only learn this handling a ton of enterprise gear.
You can now expand vdevs, but that is potentially destructive so you’d need a backup location capable of taking that load…
U.2 / M.2 add in adapters, and / or faster NICs can remediate both.
I’d say U.2 to PCIe adapter for the VM storage
PCIe 3.0 x16 can run dual 100 gig NIC’s
odd translation, but I am thinking your NVME will be VM backing storage
Keep VM’s off SATA SSD and configure backups to the big pool.