I currently have a 4 DGX Spark Cluster connected to a Mikrotik CRS812 DDQ – What Do you want to see on a 4-Spark Cluster??
I’ll also be giving >> away an Nvidia DGX spark after GTC, so you can do your own experiments. You do have to sign up and go to at least one GTC session/webinar/etc and post that to the form. Please use my link to register for GTC.
In the meantime
What do you want me to attempt to run on this cluster?
Hard for me to understand from the screenshot, but does the peak bus bandwidth imply you’re able to establish a ~200gbps connection between each Spark using this switch? (20.32GB/s*8 = 162gbps).
Which cables are you using to connect the Sparks to the switch? Is it a **400G QSFP-DD to 2 x 200G QSFP56?
Im asking because Im considering upgrading my 2x Spark Cluster to 4x or 8x using the** MikroTik CRS804 switch (4-port 400G QSFP56-DD).
This is pretty new, but NVFP4 finally is showing some promise. Would love to see some benchmarks showing different node scaling performance.
I’d also like to see some experiments with some slightly unusual network configurations
2 nodes, 2 cables
3 node mesh, does this kill tensor parallelism. Is there any benefit over 2 node system? This would be the most cost effective solution without an expensive qsfp switch.
I’ve seen limited testing of people plugging GPU’s into the GB10 using the m.2 and oculink then using a external SSD as a boot drive. A ‘budget’ DGX Station comes to mind, but I have no idea if there is any worthwhile use case for such hardware that wouldn’t just totally run on the GPU.
I’ve been using a 2 node GB10 mostly to host bigger thinking models playing around with n8n workflows.
Watched the same video. I love Alex’s videos, but it seemed like it was potentially misconfiguration on his end.
Especially after I came across a post on x where the person claimed to be able to run 8x Sparks using the CRS804 switch at full 1.6TBps bandwidth.
I know the switch being used in this post is a CRS812 which only has 2x 400G ports, but that seems more than enough to run a 4x Spark cluster at full bandwidth.
Would be great to have confirmation on this setups’ feasibility!
I would love to see total concurrent token/s on quantized but high-performing models for agent swarm applications. Like the Alex Ziskind YT video - I Thought DGX Spark Was Slower… Until I Changed ONE Thing - but on a quad-spark cluster and then compare it to other hardware like Mac Studio and NVIDIA GPUs.
I’m really dubious about Alex Ziskind’s videos on AI topics lately, His 8 way GB10 cluster while cool, like his testing was bad and he just screwed around with QSFP cables for way too long.
GB10 has way more headroom for highly parallel workloads with multiple threads.
Is a fairly new site setup by Eugr who has largely been pushing his spark-vllm-docker which is kind of the gold standard for running vllm on GB10 platform right now. But you’ll notice he has benchmarks now based on concurrency.
The nature of these devices you have a speed limit due to memory speed. You can often run multiple threads of a large model at great performance per thread, but you cannot make any single thread go faster. Adding clustering widens your road, you can get more transactions through, but there is always this speed limit.
There is also some really fascinating applications using heterogenous clusters but this is all lab testing I don’t think there is any thing you can download to enable this yet. Exo is teasing this lately more and more but I don’t think you can get this to run quite yet.
Sweet! I just registered, now i can watch it live and compare notes to Wendell’s video when it comes out!
I have only recently gotten into running local AI models, so I am still figuring out how to find value with AI, so I unfortunately dont have a good answer on what to run. That new Nemotron model looks really interesting, I will be giving it a go very soon.
I have quite a bit of data on this. Not specifically DGX but H100, GBH2/300 and MI300a. Let me know what you have in mind. Most HIP stuff gets about 80% of the throughput compared to native CUDA/ROCM code but it obviously varies quite a bit. It could probably be far better because it uses the native compilers, but the “translation layer” or whatever you want to call it isn’t there yet although it’s consistently improved over the years.
Nowhere close to the performance needed for us to swap to HIP though.
Been listening to the pre show and it makes me giggle since one of my jobs for my company is detecting “AI” interactions and blocking them for security reasons.
Is anyone else having issues pulling up virtual sessions? this is the second day now that I go to join a session and it takes like 30 minutes for the video to start, so I only really get some of the Q&A, its pretty annoying. I missed several sessions so this year might just be a wash for me.
Lots of online debate around DLSS 5… I think Nvidia missed a great opportunity to highlight a great use for generative AI on background art in video games.
For instance, lots of sports and racing games have atrocious billboard sprites with low frame rate animations for the crowds/spectators. It would be great to use DLSS 5 to fill out the bleachers with more realistic people. Or how about using generative AI to create realistic grass and trees? Or use it to generate realistic crowds of pedestrians on busy city streets? What if it were applied to generate realistic “clutter” inside houses? Or enhance the textures of buildings to make them look more organically weathered?
I can see how the whole realistic face thing would be good for pro sports games where we want the likeness of a real person on the player model. Or in war games like Arma or Battlefield or Call of Duty where the idea is to try to make realistic people in the game.
I just feel like they missed a great opportunity to show a useful application of AI and instead ended up making a meme beauty filter and getting annoyed at the backlash.
I’m not sure how they could have missed such obvious applications for “filling in” the virtual world in a video game. It’s literally the same thing that they do for image editing to backfill people and background object that are edited out.
EDIT: I just thought of another great use for generative AI: making smoke look better and more realistic. Or ripples/waves in water could be made to look better too.
I would like to see DeepSeek-Coder-V2, also Kimi-K2.5 would be neat. I have been using local AI models with a 1080 ti and a apple M1. Looking forward to seeing all the results will be interesting to compare them to a friends GPU cluster.
