I’m building a similar cluster. 8U (2+2+4), High performance Ceph with U.2 and 25Gbit networking, low power (and thus mostly quiet).
Check out my build log, ongoing project with 2 nodes ready now (virtualized Ceph so far)
I mainly focused on storage and networking and I’m using AsRock EPYC4000D4U board with a Ryzen 7900. Allows for 4x U.2 and 25Gbit NIC, but no room for dGPU for that nice Intel GPU plugin.
It’s really hard to find a good board without going full EPYC 9000, but I found my niche. You really have to compromise on storage hardware to get PCIe lanes to fit in an A310. In my case, that wasn’t that important, so I got away with less.
NUCs don’t have space for U.2 drives. Which is why I didn’t consider Minisforum MS-01 or other similar mini/micro form factors. No space, can’t dissipate the heat, fans (if present) going nuts.
And Ceph really wants/needs Enterprise drives with PLP to get IOPS going. This is really the only de-facto hard requirement for proper IO. And threads, because Ceph is basically CPU bound once you deal with NVMe speeds.
2U has plenty of space for NVMe (I got a Silverstone 2U and used the 5.25" external for IcyDock backplane).