As a few of you are no doubt aware, I have been a very busy infrastructure troll. So, I will skip straight to the entertainment.
I present to you, the nGreedia 3GX: (The beatings will continue until stockholder value improves edition)
This is my third iteration of an inference rig, probably not the last, but the most effective so far.
It features:
A customized rack mount chassis straight from the factory in China ordered via Alibaba.
The CRPS power distribution board I posted in other threads, room for up to 5x 3KW power supplies feeding a cool thousand amps of 12v into 14x 12vhpwr ports connected to the BMC via pmbus for monitoring/management/load balancing.
The Asrock Rack TURIN2D24G dual socket 24 dimm board with 20 MCIO 8i connections.
Room for 8x 3-slot GPUs at PCIE 5.0 x16, a mellanox connectx-6 NIC at 200gig, and 4x T705 M.2 sticks in an asus hyper gen5 storage card. Totaling 160 lanes of pcie gen5 straight into your Neuralink implant.
From a tech stack, its just running Ubuntu 24 with:
NFS over RDMA for bulk storage, ZFS stripe on NVME gen5 drives with 350 GB arc cache for tier zero GPU attached storage, CUDA 12.9 with NCCL, and delicious Geohotz’ drivers enabling full peer to peer direct memory access between all the cards hijacking base address register 1 and riding the infinity fabric validated with the epyc system management interface live xGMI link monitoring.
This is all optimized to feed VLLM TP4/EP4 running a custom fp8 quant of GLM 4.7 most days, doing all sorts of untold evil (such as migrating a legacy end of life SAP CRM system to the cloud).
BIOS changes from stock on the Asrock board:
disable sata for faster boot times, disable PCIE ACS for peer to peer DMA, enable HSMP support for epyc system management interface access in the OS, REBAR on, pcie hotplug disabled on all mcio ports, bifurcate the one x16 into x4x4x4x4 for the asus hyper m.2 gen5 card.







