Looking at building a workstation to mess around with AI/ML/other stuff like that and want some input/opinions about what platform to go with. Things I am trying to keep in mind while shopping right now:
Keep things a bit cheaper. This will make a bit more sense with some of the build options.
Explore overclocking, but not a hard requirement.
More pcie lanes for add-in cards, multiple gpu’s, and nvme drives.
System to experiment with on bare metal that has at least pcie 4.
My thoughts/build options are:
Sapphire Rapids system with a lower core count cpu and upgrade later for overclocking. Would be the only new platform of my options.
Threadripper 3000 consumer platform to have overclocking immediately and get cheaper options for the overall build, but no options for upgrading the socket.
Threadripper Pro 3000/5000 OEM system for some options for chip upgrade with more pcie lanes but no options for overclocking.
For some added context, this is not a work system. This is just for me using as a learning platform and experimenting with hardware. I don’t have a coding/IT background professionally, so there is an emphasis on using this to learn, both hardware and software. Why I am not looking at the Threadripper 7000 is that the cost is a bit too high for what I am looking to do ( I know that’s a bit hypocritical when also mentioning a SR system for one of my options). Also, the main reason I am not looking at something like 12th gen Intel is that I would like to have more options for pcie lanes and would rather have the ecc memory option for a ddr5 platform if I was going to go that route.
Feel free to let me know if you have recommendations on what might work better or if you think one of the three platforms would be better for my wants. I know there are a lot of smarter people here than me, so I am leaning into that. Thanks in advance!
workstation to mess around with AI/ML/other stuff like that
…
a learning platform and experimenting with hardware. I don’t have a coding/IT background professionally, so there is an emphasis on using this to learn, both hardware and software.
May I ask why do you think you need a HEDT platform to learn that?
Don’t get me wrong, I understand the joy of building computers, but it seems that the thing you’re going to do (building a somewhat outdated PC) isn’t really well connected to the intended goal (learning ML)
100% fair question. It really is about getting the most out of the pcie lanes and not being restricted in case I get access to multiple gpus. I currently have a 3090 and have been throwing around the idea of having two.
Because of that as well, I am thinking of just going for a Sapphire Rapids system and keeping my eyes out for a deal on a overclockable cpu from that range. ECC is a nice to have and I would have slightly newer architecture with the ability to upgrade to a higher core count/faster/overclockable cpu down the line.
Definitely open to suggestions. All about why I had made this post in the first place.
If you keep a eye out, SR can be had incredibly cheap. Asus was selling W790 SAGE motherboards for 281USD brand new not that long ago, and a SR 56 core ES CPU can be had for ~200USD.
Sorry, this still doesn’t make sense to me. Do you know what exactly stops you from “learning ML” with the system you already have? Maybe a study plan with labs or course projects?
From what I hear, you already have a decent desktop 5800xand a 3090. That alone will provide more than enough power for hundreds of hours of practice in programming, machine learning and data processing. Lacking dual GPUs would be the least concerning.
So my 5800x3d/4070ti system is currently a w11 system that I use primarily for gaming. I would be looking to have this be a dedicated system and potentially run on linux, with the ability to completely wipe the system and start over without worry of losing data. Sorry that I did not include that in my original post. And apologies for the misleading info in my profile!
EDIT: The 3090 is currently not in a system and just sitting around!
I was looking more into the price of that board. Looks like that might have been a way to clear out stock for Asus with the new, incoming socket? I might put a price drop notification on that, but probably won’t see something like that for THAT cheap again.
Yeah that $281 price probably was a once in a life cycle sale, but with the refresh of SPR-WS, there have been a bunch of other board vendors that entered the W790 market like jwipc, maxsun and colorful which should drive the price down… however I’m not sure about ES/QS processor compatibility with these other vendors.
Yeah, there’s also the issue of availability. Haven’t seen a lot of those other boards available yet, and don’t really want to hop onto the full server boards just yet.
I did find a good enough deal for me to bite the bullet. we’ll see what happens regardless.
What about an older platform? I was sleuthing around to find out what how Tinygrad was assembling their TinyBox GPU server. I found that Tinygrad and many others were using old Epyc CPU platforms as the foundation. You won’t have overclocking since its a server platform, but I think it matches all your other requirements.
As a starting point check out the Epyc 7532 CPU (32 cores) and Asrock ROMED 8-2T Motherboard. The motherboard has 7 PCIe Gen 4 x16 lanes (6 in actuality, one lane will be x8 when you use NVMe storage). Wendell has a video on this exact motherboard on the Level1 channel.
If you go on eBay, you can also find CPU+Motherboard+RAM combos for this platform from China. There are different CPUs and RAM options, but they usally have motherboards from Supermicro instead. The main thing to watch out for is the Supermicro H11SSL-i vs H12SSL-i. The 11 is PCIe 3, so it doesn’t meet your requirements anyway. The 12 is PCIe 4 and has 5 x16 lanes and 2 x8 lanes. Basically, its “We have the ROMED 8-2T at home” . That being said, you can find almost an entire Supermicro H12SSL-i combo for price of one ROMED 8-2T motherboard. H12SSL-i and 7532 combos are at around $1000 (not including RAM).
Final note, I don’t have one of these myself. So I’m blind when it comes to vendor quality, motherboard quirks, and other possible issues. Maybe you’ll have some luck by finding other people with similar setups online.
The only reason I was looking at older platforms was that I could find a really good deal OR I was looking for overclocking. getting a whole system for around 700-1000 would be deal in this situation. that’s why I was looking at the OEM Threadripper 5000 series.
I did find a good deal on a SR cpu and w790 ace mobo, so going to see how that goes. If not, I will probably look more into the Epyc platform like you suggested. But this would be a full desktop replacement, so I would want to do a bit of research first.
