Proxmox, Ceph, Kubernetes. OpenStack if I feel very ambitious. But I migrate NAS and VMs I have over to it, so has to be maintainable
I probably buy a couple of Connect-X 4 and get a case with good airflow (3-4x 80mm fan row in case of 2U). should be able to cool the stuff. Spec is 15W for the NIC and my X550 10G chip is just as hungry and standard case fans do the job. Everything connected via DAC, so no modules initially.
not interested in 100G. too costly and I don’t need it. Maybe a card for my Workstation, but 25Gbit is best latency and fairly cheap when talking switches. It’s mostly about IOPS, not throughput. 3-4 nodes with 2x 25Gbit can probably get to 10-15GB/s sequential reads, but I have no use for it other than posting fio results on the internet.
Hey, what build did you go for in the end? I’m looking to do something similar. The hard part is getting cost down to avoid blowing the budget, and having a smaller footprint with adequate cooling.