I’m big into home servers and giving Datacenter equipment a 2nd life to avoid paying for cloud storage and services. Examples of items I’m currently using that used to be in a Datacenter;
-Xeon E5 2696v3
-32GB ECC DDR4 x4
-LSI MegaRAID SAS controller & 8x 4TB SAS drives
-SanDisk 6.4TB PCIe NVMe SSD for VM’s & caching
-10GB NIC
-Quadro P2000
-10GB SFP modules
Seeing all these new GPU’s from Nvidia and AMD, they don’t look like traditional GPU’s, that said, is there a way to interface these with traditional tower or rack style server chassis without the full rack config as shown by the manufacturers? Or will these just end up as e-waste once the industry is done with them?
Everything can be made compatible with anything, eventually. Sheer will, some 3D printing etc.
Those full rack appliances have boards in them, so even if they are not compatible with standard rack sizing or ATX layouts; it can be made compatible using 3D printed adapters etc.
As to the software stack, all in due time. If not already, great progress (in llama.cpp, tinygrad and Lemonade) is being made to utilize all compute power available; be it NPU, CPU or GPU.
When SoftBank is finally deemed to be ENRON 2.0 (or 3.0 if you count FTX?), and OpenAI crumbles as result, resulting in the entire AI bubble popping; homelabs will have access to all this on the cheap. .. and probably society crumbles to a complete halt, and we’ll have Weimar economics on global scale. Fun. Or the next generations, depending on money printing shenanigans by the Fed and ECB
If true, then for some of us with limited space or trying to keep things tight, it won’t work. 3D printing or not. I currently have my whole build in a slightly modified cheap ATX case for 2x Chinese eBay 4 drive bay caddies.
Everything is pretty plug and play. But if these newer Datacenter cards don’t work in traditional slots w/ or w/o a 3D printed fan adapter, then they won’t work for say, in many home server builds.
And can be adapted. The biggest problem is that the newer dc hardware is not 12v to the GPU anymore, its 48 or 54 volt which is harder to adapt if all you have are ATX PSUs at hand.
Yes, I recently watched a video where they used one of these with a first gen Nvidia P100 16GB I believe (for $100ish), and it did decent for LLM’s against a RTX 3060 GPU.
