Want to upgrade RAM in the rig, but price is important.
I need more RAM , but I can’t have SLOW ram - RAM speed is also important
Looks like I don’t need 128gb ram, but 32gb is really not enough for my pipelines. System is often uses SWAP.
I HAVE RIG WITH SPECS:
mb: ASUS X399-A prime,
cpu: tr1920x
PSU: Seasonic PRIME TX-850, 850W 80+ Titanium
RAM: G.SKILL TridentZ RGB Series 32GB (4 x 8GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model F4-3200C16Q-32GTZR
USAGE SCENARIO
I configured this rig in numa mode. Time to time, I use numa0 for PCI passthrough and Windows gaming. It’s the only reason why RAM speed is important. Other time, I use this PC to write and debug kubernetes operators(launch locally kubernetes clusters), and CI/CD pipelines (a lot of containers and VMs). For second goal RAM speed is not important, but RAM amount is really important
MY RESEARCH/IDEAS:
Is it possible to attach all current 4 RAM chips to one numa slot, and buy for another numa slot another 4 DDR, but different? Maybe 64 gb kit? And have mixed memory? Than I’ll have totally 96Gb RAM. In one numa node I’ll have 64GB, and in the another 32Gb. Or is it stupid idea? Can I launch in this case all DDR modules in the maximum speed or it’s impossible? It depends of XMP profiles, and the last BIOS updates of cource, but can it be different? I mean, XMP profiles and speed of sets of RAM modules? I seen this doc: dlcdnets . asus . com/pub/ASUS/mb/socketTR4/PRIME_X399-A/PRIME_X399-A_memory_QVL_report_20190514.pdfAnd there are no examples of mixed types of memory. Also Asus probably tested configuration not in the numa node (not sure if it’s important). Based in their test, looks like this mobo can’t support PC3200 XMP profiles for 8 memory chips, am I right? Or they’re too pessimistic?
Is it good idea buy different kits in amazon, check them, and return back if they’re not compatible?
What happen, if I use 2 RAM sticks in one numa node, and 4 in the another?
Is it possible to have big speed in one numa node, and more RAM, but slower, in the another?
There are nothing in the MB manual about weird RAM configurations. Nothing about NUMA or UMA modes. I think, ASUS engineers tested this MB in the UMA mode only
Still don’t understand how it works. Could you please explain?
If I understand correctly, what they wrote in this document, I can use 4 DIMMs in the quard-channel mode (UMA), and what above(additional RAM sticks, up to 4, totally 8), the system put in the single channel. So, system has, in this configuration, in one time, part of memory in the 4 channel mode, and in the same time, part in the one channel.
TL;dr: 4 DDR in the 4 channel, and + up to 4 in the single channel mode, totally up to 8 DDR sticks, and first part of them works in the full speed.
Am I understand correctly?
If yes, how it could works in the NUMA (two channel) mode, and which configuration should I prefer based in my goals?
If I understand correctly how it works, I need probably something like this:
1) 2 sticks: 16gb + 16gb PC3200 (or even more fast?) to install them in the D1 and C1. This is numa0, and it’s two-channel memory. They should be from one vendor, one memory kit. 2) 4 any sticks in A2, A1, B2, B1. Maybe my current sticks, or maybe better to sell them and buy low-speed but with big amount of RAM. It is numa1, and It works in the one chanel mode. There are no any specific requirements for them, excluding which I wrote below in the “3)” 3) all 6 sticks should have same level of CAS (latency)
Is it sounds realistic? Like possible hypothesis? Or maybe I read a doc and understood it incorrectly in some cases?
As you increase total memory amount to its hard limit, the workable speed will be dialed back, away from it max. applicable speed [nature of any memory controller]… If not mistaken for TR Gen1, its a backpedaling of 2666 ceiling, towards 2133?
Look also at memory producer, if memory been tested on your mainboard.
That sounds more an issue, should you populate uneven memory modules [ex. having multiple 4GB + 8GB sticks populate the board]. Just like when pairing same sized chips, but one set is at slower speeds, the common memory speed across system matches slowest. Having uneven memory amount per dimm, would only accommodate multi-channel rating, for the common matching amount [ex. 4of8GB], leaving those leftovers at single channel.
If you believe the AMD promotional material, each numa node inside the chip has access to four dimm slots across two channels… similar to bigger brother dual SP3 socket servers, where each socket has four channels and eight dimm slots.
I wouldn’t bother mixing memory capacities, since you lose bandwidth. You’re also not going to be able to mix speeds, despite the independent channels. Go with same size sticks, keep it in dual channel mode for each node.
Guys, you’re wrong. It’s technically possible to get not-same memory and to not lost dual-channel mode and, even, speed!
I performed several tests with different memory sets. Sorry for long time until it happen - unfortunetly, I’m not sort of handy person. I always felt fear to go inside PC and touch something. So, I waited a free time of my husband. He just bough upgrade for his RAM (2x16gb 3000Mhz) in order to replace his old set (2x8Gb 2133Mhz). He was bored to attach/detach RAM modules in ways how I said him, and entire test took about half of hour, but me (we?) finished it
These are all photos of my test: imgur . com/a/m9eoTI2
you can launch different memory kits in the threadripper rigs
in the complicated sets like: 2 x 16gb PC3000 + 4 x 8 PC3200 and get a dual-channel memory, launched in the 3200Mhz (how? The first kit doesn’t have XMP profile for PC3200, only for PC3000)
Memory in the different numa nodes always launched in the same speed and the old memory can force your rig to work in the slow mode
To build mixed RAM configurations in tr4 rigs, try to use online stores which permit return/exchange items
if you install 4 DDR in one numa node, it doesn’t affect another one. It can launch second in the two-channel mode with two other DDR. Official information and suggestions in which slots you have to install RAM modules, is complete wrong. Real situation is much better than described in the official manuals.
Questions:
How is it possible to get all RAM in the PC3200 speed, but not PC3000?
Can I be just luky in this test, and another RAM (not G Skill for example) can be just not work as good as it worked in this test?
Based in this test, which RAM do you suggest to this upgrade? Want to buy something in the incomming boxing day.
