I tried roughly a dozen distros on my system and so far opensuse leap 15.6 with xfce desktop seems to be the best blend of not being buggy and having good performance.
A bunch of the distros were just broken on this hardware.
I liked Gnome 2 but the project has become a dumpster fire as of late imo.
So I did download Ubuntu 24.10 and I am running into the same behavior where all core load downclocks the CPU to 2.6Ghz and power limit appears to hover around 240W. Let me fiddle with the EUFI a little more but Im losing hope in Seamicro.
The down clocking to 2.6GHz is normal for all cores being loaded, 36 cores loaded should run at 3.0GHz, 32 cores loaded will run at 3.2GHz and 28 cores loaded will run at the full 3.5GHzā¦ atleast thatās the behavior my asus board is showing.
Ok, youāre absolutely right. at 28 core I can pull 300W consistently and the actually get higher token/sec
Thatās even stranger, we ALWAYS have wild performance degradation running Windows 10 and 11 Pro on enterprise server skus.
Workstation is only recommended when you are runnin an oem install.
We run Winder server when windows is required.
This may be the root of the evil here.
Iām curious about if you tried DeepSeek 671b or 70b Q8 by now?
Hi nice Thread topic here.
I am on a intel system of similar generation on the 4677 platform (but no hbm yet).
Some pitfall i stumbled over. And maybe it can help you and others.
When playing around with bios settings, there are many of combinations.
Some are a bit counter intuitive like AVX powerlevel (usually nominal, level 1, level 2) ā Keep it nominal, level 1 and 2 slow things down, they are for thermal control.
Another thing that can help is the Power duration limits.
In windows critically important the performance profileā> Performance or even better ultimate performance cmd or powershell as admin and : powercfg -duplicatescheme e9a42b02-d5df-448d-aa00-03f14749eb61 (But this typically gives terrible high idle power draw, but you can turn it on off when you need it).
And for ollama try it with SNC=Disabled.
I also received an offer for 9480 Max, and i would be very curious to see some benchmarks on llama or ollama models with these. And correct settings.
Sorry to necro this thread, but the 9480 hasnāt been discussed in the forum since here in March.
@monstercameron or @twin_savage, did either of you get the config/settings on your 9480 systems dialed in enough to run large models to your satisfaction?
What was the final verdict on these? Are they worth the trouble?
Thanks.
I never did get it running as fast as it *should be running; I chalked this up to the super-unoptimized state of opensource CPU inference that we find ourselves in at the moment.
I did want to try the cache aware scheduling RFC to see if that remedies the issue but havenāt gotten around to it yet. It looks potentially very promising to reduce fabric contention.
https://lore.kernel.org/lkml/[email protected]/#r
I also wanted to try the last version of Clear Linux to see if it had any special sauce in it. Phoronix benchmarked it on an HBM equipped Xeon and it scored triple what Ubuntu 25.04 could in alot of the memory performance benchmarks.