Return to Level1Techs.com

Intel or AMD TR for AI and ML?


#1

Hello!
I need a good base computer for AI/ML, Tensorflow and general CUDA compute.
This thing is going to run Linux so I dont want to accidentally pick a motherboard that needs special drivers.
Drivers and ease of install is probably the most important thing in this build.

Starting with 1 RTX 2080 or 2070 (no nvlink…) but with room to add more GPU:s so I guess I need good amount of lanes on the MB.

My first thought was a low end TR because it has a lot of lanes to the MB. But maybe a higher end Intel is better if I want an Intel Optane for cache?

Budget is in the “good value” category :slight_smile:


#2

I think if you get a 900p you can use that as an SSD cache even on an AMD system, since it is all done by the kernel.


#3

Well the thing with intel x299 HEDT platform,
if you want or need more then 28 pci-e lanes, then you are kinda stuck to the higher cost core i9 cpu’s starting with the 10 core 7900X and up.
Those cpu’s are significantly more expensive that AMD’s 2920X 12 core or 2950X 16 core cpu’s that AMD is currentlly offering.
You also get more pci-e lanes like you mentioned aswell.
TR seems to perform very well on linux.

A more expensive intel i9 X299 setup could be worth it,
if you trully would benefit from its overclock ability, improving your workloads significantly.
But even then its the question if it would really be worth almost double the money for just the cpu.
I personally don’t really think so, but i dont know enough of your specific workload field.
To really judge about that.


#4

I’d do TR myself. No possible way to run outta lanes.


#5

Threadripper. It’s substantially less expensive and I don’t think Intel will be trailing ahead in performance much for it to be worth.


#6

Off topic; is that an oxymoron?


#7

For ML/DL, the advantage seems to be in Intel’s favor due to AVX512. https://www.pugetsystems.com/labs/hpc/AMD-Threadripper-2990WX-32-core-vs-Intel-Xeon-W-2175-14-core---Linpack-NAMD-and-Kernel-Build-Time-1303/

You’ll want to check if your workload benefits more from CPU parallelized tasks (advantage: AMD), or if parallelization won’t scale as much and if the higher single/few core performance of Intel is better for your use case.

On the GPU side, AMD’s ROCm is improving nicely and packaging installation ease is supposed to get better, but for the time being NVIDIA’s CUDA mostly offers better and more consistent performance and ease of installation: https://www.phoronix.com/scan.php?page=article&item=rocm-20-linux50&num=1

Of course if you’re thinking about vendor lock-in issues, you may want to support AMD, but for the present moment NVIDIA/CUDA the performance winner.


#8

I thought we were comparing Intel’s Extreme Editions to Threadripper no? Then again, I probably would much rather buy Threadripper myself.


#9

@DrewTechs
I was trying to say that my reply was off topic;
trailing ahead
trailing = behind
ahead = forward
It doesn’t make sense to me, sort of like government intelligence, an oxymoron.


#10

Threadripper. More lanes. Upgrade path. Works with optane anyway.


#11

Indeed ML / AI is very dependant on the toolset you want to use.

Look at benchmarks with your required toolset and buy want performs best.

Zero point buying a Radeon VII with insane Open CL if your tools use Cuda.


#12

That could make a 7900X 10 core interesting maybe.
The only disappointment with intel’s skylake-X X299 platform is that you need a core i9 to get the full 44 pci-e lanes.
But if avx512 impacts performance significantly in his particular workloads.
Then it might be interesting.
But then we need some benchmark comparisons between a 2950X / 2920X and a 7900X.


#13

Need to look in to that AVX thing and ask the dev:s more specific what we want to run on it.
Looks like they want RTX GPU:s for CUDA so AMD graphics is out for now.

My understanding is that its going to be used primary for GPU load so CPU should not be that important other than expandability, lanes etc.

I personally want to go TR for supporting AMD and I have never built one :slight_smile: But I also need a good selling point for it I guess!


#14

Well yeah i understand that people want to give AMD some love,
Because of the back fight they have done against the big giant intel.
But of course wenn it comes to your own money, you should always look at it,
from an objective perspective.
you should always pick what gives you the best performance for your money.

Of course AMD’s TR offerings are hard to deny at what you get for the money.
So we basically need to see more benchmarks to compare several cpu’s.


#15

The way I see it, AMD usually have two key advantages for ML these days:

  1. Better Price / Performance ratio. This makes them a better choice hardware-wise for the budget-minded, but the more expensive RTX will still have better performance in the end. And Intel CPUs still has better single-threaded performance.

  2. Better OpenCL and Linux support. If you want to create a multi-user setup where people can send batch processing to your PC (say you are part of a faculty at a University), Linux makes this extremely easy to set up, and is rock solid. It also allows you to run headless, meaning you do not waste any cycles on the GPU for displaying a desktop. So if you want to squeeze every last drop from the system, AMD sure helps here.

However, CUDA is the performance king at the moment, no disputing it. OpenCL might reach 80-85% of CUDA on a good day.

Another thing you might want to consider is to get a beefy FPGA. When it comes to Performance-Per-Watt, nothing comes close to beating it, which sure helps if you want to put that thing in a mobile battery-driven unit. On the flip side, it’s an extremely low-level language, so it will take a lot of time to get a neural network onto it. Depending on your use case, an FPGA could be a good alternative.


#16

I like FPGAs like noone else, however a “beefy FPGA” is likely going to cost more than the rest of the system.
The benchmarks Xilinx released for their Alveo U250 show very low latency (critical for purposes such as self-driving cars and google results), the problem is all software needs to be optimized for int8 operations.

Anyway, link to Xilinx Alveo U250.


#17

You are correct of course, I was thinking from a pure “What could be interesting for an ML/AI application” perspective. :slight_smile:

But yes, a good FPGA will cost you an arm and a leg, and perhaps your shirt as well. The tech is simply too expensive to be worth it today, and needs to come down in price for mainsteam appeal. The possibilities sure make you drool though, imagine running a performant ML system on a 9V battery… :drooling_face:


#18

Of course it is interesting for ML, you basically have hardware that can change to be the synapse you need. A hardware implementation of ML/AI software so to say.

Powerdraw is that of a GPU (225W in case of the U250).


#19

This is true if we look at the top end performances, but further down the spectrum, I did some research on the efficiency of FPGAs a couple of years ago. They actually usually reach around 50-60% of the GPUs consistently, with 10% of the power. Of course I only did some preliminary research before we concluded GPUs were better for our purposes (FPGAs are still not great at floating point arithmetic), but, yeah.

It’s not a clear-cut scenario though which one is better. If you need tonnes of parallell processing and not that much FPO but quite a bit of logic gates (which is great for signal processing), then FPGAs are great, but for other use cases, not as much… :slight_smile:


#20

We have an NVIDIA Jetson that seldom gets used, I dont know how that perform compared to a RTX