RTX Pro 4500 x4 or A100 40GB x4 (decision in the morning)

Hardware Hub
,
1

Time is of the essence. Who knows what tomorrow will hold? RTX Pro 4500 x4 ($3,299) or used 40GB A100 x4 off eBay (around $4,000/ea).

The RTX Pro 4500 means I have to go the route of getting a PCIe switch, inviting a fair amount of complexity.

A100 means I just need to find NVLink bridge adapters and hope I’m not buying a GPU from eBay that’s on its last leg.

Those are my frantic thoughts after letting the $10k RTX 6000 Pro deals expire in my cart last week. Any feedback is appreciated. What am I missing here? What am I not seeing?

2

Apparently, I’m not the only one who had this idea…. CoreWeave proves Nvidia's aging AI GPUs from 2020 can generate profit nine years after deployment, signs A100 contracts into 2029 — power constraints and legacy infrastructure keep old GPUs profitable | Tom's Hardware

3

I’m weighing the same tradeoff between the PCIe-switch complexity of four RTX Pro 4500s and the age and NVLink requirements of used A100s. The A100 option seems attractive for memory capacity, but the condition of four second-hand cards adds a lot of risk to an otherwise straightforward build.

4

Exactly. The second hand issue is the real problem.

Hell, this is for enterprise, so I’ve considered getting SXM4 x4. You can get x4 with the board for less than $10K. The problem is it seems the chassis costs just the same.

5

Any input, at all?

6

Since this is for enterprise use, I’d favor the A100 route only if all four cards and the chassis are verified; the PCIe-switch path adds another failure point.

7
  1. Is this purely for inference?
  2. What about nvlink?
8

I am currently in a similar position deciding on what GPUs I choose for the future.

Screenshot_20260820_104119_Sheets
Screenshot_20260820_104119_Sheets1728×744 265 KB

I made a spreadsheet to compare all the options. Prices are from Ebay in Euro and may change daily. But they give a ballpark at least.

Of course, I didn’t include the A100 yet. But regarding performance indicator of Techpowerup it would be relative 78, well below the reference RTX3090 with 100 and the RTX4500 with 137. On the other side internal memory bandwidth is nearly twice as high, which is good for inference. So I would assume that inference performance is similar. But the A100 would certainly win regarding training.

So there may be additional factors:

  • available PCIe slots
  • PCIe Gen4/5
  • Total memory size
  • Support for NVFP4
  • Future driver support for Am
  • Additional cost for Gen4/Gen5 switch
  • Cooling

For me there are fewer variables. I have a EPYC system with 7 slots and need no extra switch. As I am on a lower budget I target 64 GB memory and will probly choose 4x RTX5080 as best compromise. Cost per GB and per performance is lower than for 2x RTX4500. And 4x RTX5080 has about twice the performance than 2x RTX4500. I would buy the 4500 only in the single-slot version. YMWV.

Currently I run Qwen 27B Q6 with tensor split on a RTX4080 and an old RTX6000 Quadro with 30 - 50 t/s. I hope to double this at least. The speed should be well above an RTX5090 or even a RTX6000 Blackwell, but for much less money.

Of course, further expansions are possible with either a Gen4 or Gen5 switch. For inference a Gen4 switch should be ok. With cables it would be about USD 1000. The cheapest entry into Gen5 would be 4x MCIO 8i switch for about USD 1,200 - 1,400.