RAMageddon - how to plan around it?

Honestly,

Short of a softening in the AI market, I don’t see memory getting any cheaper any time soon.

Maybe as AI workloads become more and more demanding requiring more and more RAM, older and smaller sizes of RAM will become cheaper on the second hand market, but that really sounds like no fun.

We used to talk about the “Chips Act” and it helping to bring more chip manufacturing capacity online to help fight the shortages in the market and bring prices down, but it hasn’t been as effective at creating new capacity as we had hoped, and most of the capacity that has come online thus far has just instantly been gobbled up by the insatiable appetite of the AI industry.

I don’t think we can even hope for affordable second hand DDR5 when DDR6 comes around, because the demand for AI is so high that most aren’t retiring their old systems when they get new ones.

How to plan for it? Only way I can think of is having bought extra RAM back in 2022.

There would have to be an absolutely cataclysmic event in the AI industry for this trajectory to change enough for there to be a substantive change.

I think optimization in other workloads than AI such that we need less RAM is more likely.

For a long time now the mantra in software development has been that hardware is cheap, and software developers are expensive. To this end they have often used highly wasteful high level languages, relatively unoptimized code, and piling on bloated libraries to avoid reinventing the wheel, but there is a small movement that is gaining traction proposing going back to basics on coding and writing more things from scratch, and optimizing for RAM use and CPU cycles.

Casey Muratori - among others - has had many talks and become quite outspoken on this topic online. It is worth listening to some of his talks on how the software industry has gone astray.

Software engineering is very much controlled by business interests. There’s no time to optimize, and it doesn’t take too long to realize that crappy unoptimized RAM and compute hogging software benefit cloud providers… all of this benefits those who own the platforms.

All this “growth” is fueled by debt and speculation, and will eventually collapse. Consumer RAM will not collapse in price, because there’s a shortage now, and there will be then. This will keep hurting the second-hand market, because there’s less hardware being sold now.

Surplus equipment after a collapse will be coming from datacenters, which I suppose will be fun and cheap for homelabbers, as it was after the dotcom bubble popping. I’m pretty sure companies will keep prices inflated, because we have monopolies and cartels now, and that’s what these do. So this will last a while.

I’m used to running decade old hardware so I’m not overly concerned about this.

But a lot was invested in manufacturing capacity, so that will eventually bring prices down. I think the cheap hardware we had before the AI craze was because of excess manufacturing capacity available after the cloud hype plateaued.

I don’t see AI bubble lasting much longer. Yes, AI is here to stay, but current insane growth is artificial - there is no benefit to pay for it.

Even Wall Street now realising that.

On top of that, there is a massive production capacity growth everywhere, which is to kick in 2028.

IMO this means at the Zen6-time upgrade will become bearable, around Zen-7 time it will be quite interesting, around Zen8-time (2030) a no-brainer.

For me, if there is no intermediate breakage, I think I’ll be doing it around Zen9-time (“Nemesis”). Around then, PCIe6 or even PCie7 should be everywhere, along with 100GbE Ethernet, DDR6/LPDDR6 on CAMMx should be fast, cheap and ubiquous, CQDIMM should be here t double per-module capacity AND bandwidth, x86 will get massive tensor, matrix support and APX ( dobling the CPU GPR, 3-operand instructions etc) , 3DVcache will be part of every package on every die, RDNA6/UDNA2 should be here, AV2 and H267 should have HW en/decoders, and AM6 socket should be mature.

And by then, perhaps having a potent iGPU in the package, along with separate LPDDRx on the package, along with its separate IMCs and perhaps 4 external memory channels might become a new consumer high-end option.

1 Like

The earliest forecast return to normal pricing is 2029. To keep track of this yourself, read analyst (e.g. Trendforce) reports and RAM manufacturer (e.g. Micron, SK Hynix) earnings call transcripts.

Hardly anyone is talking about DDR6 right now.

RAM performance is only 1 aspect of PC performance.

This probably leaves Zen7 in the game (if DDR5 prices fall by then), but that one is also expected on AM5/DDR5.

4- and 8-channel RAM configs exist. Workstation CPUs (e.g. TR, Epyc, Xeon) say hi.

The least expensive plan is always to not buy anything. If you’ll need an upgrade between now and 2029, buy now because prices aren’t going to get any better between now and then.

Definitely wait if you can, as things are changing super rapidly. I wouldn’t sink money into Zen right now unless your current rig is uncompetitive. Nvidia’s DGX and RTX Spark are unlocking whole new computing models BUT per Ian Cutress, customers are asking Nvidia to crank out x86 CPUs to match their existing tooling.

Where I’m going is right now Zen is the king of gaming and HEDT, but if games and other apps start leveraging the capabilities of Spark local AI with vLLM (which uses CUDA and is much faster than competing solutions) and 400 Gb/s clustering, Zen could have a hard time staying relevant.

1 Like

Sure. This is why I’m looking a couple of years ahead. By then, it’s obvious that something will be needed. I guess CQDIMM (= MRDIMM equivalent for UDIMM - uses the same bag of tricks) could do for the Zen6 and maybe Zen7 time.

They may say “hi”, but I’m not answering that. besides, there should be 32- or more-cored versions for consumer socket by ZEn7 time. All those cores should e quite a bit faster than existing ZEn5 cores (so demanding more RAM bandwidth to breathe comfortably) and there will be many more of them. Plus probably powerfull NPU).

Even at present time, cores in 9950X can be often tight on RAM bandwidth, which is going to get worse as core potency and number of cores is rising faster than bandwidth, which is stagnant.

How do you know that ? To me it looks like we might be over the peak, if not now then soon.

Markets are slowing down. Banks aren’t to keen of covering that boom anymore.

1 Like

Then you’re ignoring a huge part of the current reality of personal computing: that traditional consumer-oriented uses cases, e.g. gaming, are no longer core business for mainstream hardware players. You’re right that RAM bandwidth is a limitation, but DGX Spark, RTX Spark, and Quad + Octa-channel socketed x86 systems get by just fine on the workloads for which they’re optimized with DDR5.

The 5090 and 5080 dropping off the face of the planet with no replacement for the 5090 should tell you how important consumer-oriented use cases for that hardware is going forward.

In short, local AI is the new high end gaming. That’s not going away anytime soon as it’s a higher margin use case (read: buyers are willing to pay more for it).

I answered that in my previous post. TechPowerUp has the best coverage of the sources I mentioned.

I hope you’re right, but again even if datacenter demand collapses, local AI workstation, e.g. Nvidia Spark and Strix Halo are higher margin than gaming and are attracting considerable retail consumer interest. There are quite a few higher income people who don’t game or do traditional content creation but are interested in local AI, and those people can outspend the average consumer easily.

1 Like

What’s wrong with ignoring the parts that are irrelevant to me ?
What I’m saying is that CONSUMER STUFF will get much faster cores and many more of them.

Which means it will have to find RAM bandwidth to feed them.

That has nothing to do with TR and EPYC stuff on 4/8/12/16 RAM channels etc etc.

None of them seems authoritative source. I find latest beahvior of market, main players and banks more convincing.

Because that’s ignoring the facts? Market segment changes don’t stop or cease to exist because you ignore them.

AI is consumer stuff now since DGX Spark hit retail availability as it allows folks at home to run the exact same stack datacenters do. AI is also higher margin than gaming and traditional content creation. As such, it will dominate the consumer market going forward.

FTR, Nvidia DGX Spark uses 16 memory channels. So yeah combine that with local LLMs becoming the major driver for consumer computing and there’s not much immediate need for DDR6 and dual channel RAM is likely to become a legacy or niche config.

Past performance is a great predictor of future behavior.

For example if AMD didn’t come up with the big advancements they had multiple times through the decades, it is extremely likely Intel would have consumer platforms topping out at 6-cores right now. That is a common sentiment, however true it may be.

Point is we won’t be seeing massive leaps because there is no major developmental pressure for normal consumer hardware, especially with consumer demand at all time lows due to price, production of consumer destined NAND & flash memory being low (several years ago those manufacturers over produced and were selling things for cheap, they changed course, now those companies are literally making record profits, in many cases they have more profits in the last couple of years than EVERY year they operated for before those couple of years)

They have money flowing in, especially for AI oriented hardware, any new and faster or significantly improved “consumer” hardware will be like the RTX 5090, out of stock or going for $7-10,000 new…

If you need it & can afford it then buy it now. Otherwise wait, but it will be a LONG wait for prices to drop to earlier 2026 levels, inventory to improve, and hopefully performance is half decent for the money (but I kinda doubt it).

The market pressures would be a lot different if local & enterprise level AI wasn’t extremely capable and useful in ways that no other software has been. The simple fact is extremely few “consumer” workloads need 5.0 X4 speed NVME drives, or 24 cores, or 16GB+ of VRAM powerful GPUs or 64+ GB of fast RAM, more than 30 PCIe lanes, typical heavier consumer loads are just video games and light content creation (latter can easily be done on a super efficient modern M series Mac laptop or mini computer), those loads aren’t putting any real pressure on advancement for consumer hardware.

Enterprise trickle down is a real thing, but it won’t be quick, affordable or truly consumer oriented. We had several years of affordable decently well performing hardware where people could upgrade “just because”… Now it’s “upgrade if I have no other choice”. It may be 4-5 years before we see anything resembling the good times but with inflation eating your buying power away.

1 Like

WTH are you talking about ? 9950X exists.

16 fast Zen5 cores on 2 DDR5 channels. Per-core bandwidth might be tight for some apps, but it is obviously good enough for many others. People do buy and use them.

Zen6 is coming. With 30% faster cores and 50% more of them. So with all else being the same, each Zen6 core would get LESS bandwidth, while having MORE need for it.

With Zen7 on the same two DDR5 channels, the same problem will become far more dire.

Which means they would become inadequate as an upgrade from 9950X for those applications.

So my question was, what is AMD about to do about it. And using TR or EPYC clearly isn’t the answer as these are in whole other segment AND ZEn6 will clearly have 24-core Zen6 and plausibly Zen7 32-core on the same AM5 socket and thus on 2-channel DDR5.

So all those “facts” outside AM5 are irrelevant for the segment that I live in.

“Cars are irrelevant, I ride horses! There’s a faster mule cart coming next year, what is going to be done about it???”

BTW I’m not cheering any of this on. But consumer dual channel RAM use cases are headed for a long winter. AMD don’t make RAM so there’s nothing they can do about it. Except delay Zen 6, which they already have. The Zen 6 SKUs that receive the most attention will likely be Strix Halo’s successor as Nvidia are straight up lapping AMD with Spark.

Stepping back a bit, the fact that Intel basically stopped releasing Battlemage gaming GPUs after spending a small fortune to get into the market says a lot too.

Strawmanning my arguments, as usual.

“Car” (TR/EP) is can often be no faster or SLOWER than a “mule” (9950X). It has lower sustained turbo, lower RAM frequencies, less overclock margin etc.

And BTW, RAM pricing $/GB for the “car” is TWICE AS INSANE as for the “mule”.

If one is in that niche, why spend unneccessarily ?

RLY ?

And how would going for a “car” with its many more RAM sticks ( each one being at least RDIMM if not MRDIMM and thus MUCH more expensive) solve that exactly ?

Let me put it bluntly for you. You’re asking about a use case none of whose big players cares about anymore. The writing is on the wall everywhere:

  • Zen 6 delay
  • Intel Ultra 9 290K Plus cancellation
  • 5080 and 5090 EOL
  • No B70 Battlemage
  • No 128 DDR5 max speed DIMMs
  • No true AMD 5080+ equivalent
  • No PS6 and Helix release date

AMD isn’t going to do anything about it because they don’t care. Same with everyone else. They already make more money doing other things.

High performance dual channel is dying right before our eyes. It doesn’t matter whether that segment is your focus or not. It’s not the focus of the people who make the products for it, which means it will either stagnate or die.

Chinese companies might fill the gap.

I don’t think so.

But I’ve bookmarked it, so we can revisit it later.

Well as long as the industry has high demands like for the ai data centres.
Memory manufacturers can ask whatever they want.
This is just the chance they have been waiting for.
Nobody really cares about the home pc enthusiast market as much anymore.
Because that market is shrinking for years already.
They can quadruple their profits so manufacturers simply shifted their focus.
Nvidia, Intel and AMD have also done this as well.
It is a bit unfortunate for us pc enthusiasts but i don´t really see this changing anytime soon.

2 Likes

In short term, yes.

But all that industry is deeply dependent on the mere mortals. AI needs human produced data to process. And it needs human customers on the other end.

it won’t get that if people don’t have access to personal computing etc.

1 Like

Well… yes but that is what smart devices like phones etc are for now a days.
Those devices are relatively cheap to produce and almost every human has one.
The home pc enthusiast market like gaming pc´s etc is shrinking for years already.

1 Like