Hello from South Korea

Communities
1

Hello all!

Glad I found this place, I had seen some of Wendell’s videos, which were great, but never knew it had a thriving community behind it.

I primarily am seeking out places like this because I’m considering building my first Xeon-based build.

I have gotten my hands on a pair of Xeon Gold 6240Rs (Cascade Lake) and ample DDR4 ECC memory to go with them, and have seen mixed answers on whether or not its worth getting the rest of the parts or a shell of a workstation to put them in. Currently my home machine is a 10900k-based platform, and my daily driver OS is Linux Mint.

Any advice on an area of the forum I should head to is appreciated!

9 Likes
2

Welcome to the forums! Are you planning on doing llm inference with your setup?

1 Like
3

Welcome to the forum, can you tell me what it is you are wanting to do with the machine? Homelabbing, AI, software development, gaming?

1 Like
4

Thanks guys.

I would like to try LLM inference for sure, haven’t attempted it yet.

Primarily home lab, most likely networking focused, virtualization, productivity, some light/moderate gaming.

1 Like
5

So I looked it up and the Xeon Gold 6240R is about 25% slower then a Threadripper 5965WX, mostly because of lower clock speed. Overall I would say it still usable for a workstation build. It does have 6 memory channels, too.

Though you should be aware @cescudero95 that if you do inferencing on the CPU with the model in main memory it is still going to be slow. It would likely be acceptable to use for a single user but not more then that.

If you’d add GPUs to the build though it could be a different picture. A 24GB or 32GB GPU would be the minimum, then you could put the dense layers in an MoE model on the GPU and the sparse layers on main memory. This should make it useable for experimentation and single user use-cases.

If you put in more GPUs and focus on inferencing on GPUs with the model in VRAM these processors should be more then fast enough in my option.

You said you got your hands on a pair, so do I understand you want to build a dual socket machine?

For any questions how to start with LLMs let us know.

1 Like
6

Yeah, that’s right, I have a pair of the 6240Rs

I was recommended the C621E SAGE motherboard in that regard, is that solid? Any competing recommendations for my use case?

I would also aim for at least one solid GPU. The Xeons do not have iGPU anyway, so have to have something.

7

I have no experience with this motherboard but it looks like an okay fit. I think for a homelab this machine is going to be doing very well. Gives you a lot of room to experiment.

1 Like
8

Thanks, yeah I think so.

One thing I’m trying to figure out currently, it seems there are SQUARE and RECTANGLE coolers for this socket depending on the board setup. I’m trying to figure out if the Asus C621 SAGE uses Square or Rec arrangement, searching for coolers (I only have the Xeons, I didn’t get the coolers with them).