Has anyone tried to run upstream GLM-5.3 Flash on 3x DGX Spark?

High-Performance Computing Machine Learning, LLMs, & AI
,
1

I think in theory it fits in memory (3 x 128GB = 384 > 321?) and should leave enough memory for some context. Does anyone have a 3 spark cluster and tried it? What does it look like performance wise?

2

Shed loads run it over here: GLM-5.3-Flash: 320B total parameters / 18B active - DGX Spark / GB10 - NVIDIA Developer Forums

I personally run TP=4 and DSv41F. Not had a need for GLM-5.3F yet.

3

Thanks. Could you point me toward the reply in question? I’ve read the entire thread diagonally and could only find examples with 2x and 4x in FP4, not 3x with the native FP8

4

You could try this recipe or any others made by MiaAI: