I’m setting up a TrueNAS-on-bare-metal and ZFS box and I bought a lot of 12x Intel enterprise SSDs that are 480gb a piece. I’m not going to use all of them but I figured that I could potentially use 4 of them in raid0 as a fast scratch disk space. It stands to reason that this sort of setup isn’t really ZFS’s forte, nor the point of ZFS, so I was hoping to be able to set up a four-drive raid0 mdadm disk using something like ext4 or f2fs, but TrueNAS being BSD, mdadm isn’t an option.
So, my question then, is it possible to set up a VM in TrueNAS (say, arch linux) that I pass an entire storage controller into, which contains all four SSDs, and set up the mdadm array there? It’s not really clear to me how to expose the storage to other VMs at that point.
Maybe what I’m asking has some major pitfalls or something I’m not considering. Open to suggestions - at this point I’m just trying to mess around with some extra drives I’ll have laying around.
EDIT: Hmm… it occurs to me that maybe what I need is TrueNAS Scale, instead, as it seems to run on the Linux kernel, which I didn’t realize.
You are asking a tough question from FreeBSD. You are using B-Hyve hypervisor in TrueNAS Core. It should have an option to passthrough hardware devices, but I don’t know if it’s as advanced or easy as in Linux.
@diizzy you know FreeBSD way more than myself, would this be possible?
If this won’t happen, you probably want to stick with TrueNAS Scale if you prefer the TrueNAS management interface, which is based on Debian, just like Proxmox and uses QEMU+KVM instead.
I believe Virt-Manager is able to connect to a FreeBSD host and control B-Hyve that way, so it may give you some easier options to add devices once you have VT-D or AMD-Vi enabled.
bhyve does support it however I haven’t used it myself bhyve(8) (search for passthru)
The Wiki page is a bit stale and should be considered more of a scratchpad than official documentation if manpages are available
Given the above it should be doable however I have no idea how much manual work is needed in TrueNAS (ie how granular you can configure bhyve using the UI). This is most likely better answered in their own forums.
As for your performace assessment I’d do actual benchmarks as it may not be the case regarding ZFS vs ext4 and your workflow. You also have UFS(2) which probably performs similar to ext4 in general but you need to manual hacking in TrueNAS in that case to use it.
My guess is that you’re probably better off running a plain install of FreeBSD in the end though
