Interesting results. Would it be possible to test some even larger MoE models, like the DeepSeek-R1-0528 quant intended for AM5+GPU (which technically does not fit without overhanging into SSD), or maybe a smaller quant of Qwen3-235B-A22B?
Given the sudden interest on very large open-weight models, I wonder if it just had its thunder stolen for not offering a 256GB variant, if that would not throw the prices even further out of all sense, and if it is electrically possible at all.
But to be fair, at the time the design goals of the chip was probably issued, the SOTA of open-weight LLM was likely either GPT-2 or maybe llama-65B, with people clambering through AI Dungeon 3’s UI to have a taste of GPT-3.