Very interesting thread, and lots of great responses. Thought i’d add in my 2c here.
I have a Truenas box with 64GB of RAM and an Arc B60 - running gpt-oss-20b as a reasoning agent. It’s true the Intel OSS stack is bad, but it is at least stable if you use one of their supported models. On gpt-oss-20b i can easily get a solid 40-50tok/sec and with tool calls it enriching the output it can be a genuine alternative to ChatGPT or Claude in conversational questions. Going up from 24 to 32GB and more XE cores/Matrix engines would allow the B70 to get 30-40% more and i’d be inclined to go for a newer model. I am currently using the B60 to do some HomeAssist things but really it is just for testing & learning.
Regarding the 9700’s - I have just ordered my 2nd unit for my home inference/agentic coding box. With 64GB of RAM (I only have a consumer Ryzen board, so also the same 24 PCIE lane limitation) - you run into all kinds of fun research and time sink endeavours, such as evaluating vllm vs llama-cpp, layer split vs tensor parallelism - etc.
My take is - if you want to go deep and learn the guts of how AI works, and you can justify it - eg it will help your work/job or ability to earn money - go dual GPU’s and learn as much as you can. At a recent work talk our GM said that “we will need everyone to be AI specialists” - and i think it is true for everyone in the tech industry. We can all specialist in storage, compute or whatever, but having deep AI technical expertise is a future commodity that is worth spending time and effort growing into.
Anyway, getting myself back on track - Dual 9700’s has allowed me to get around 60-80tok/sec with tensor parallelism on a few of the Anthropic distilled models on Huggingface such as Qwopus etc - and I use Forgecode as my harness on them to generate real, useable code. AMD’s software stack is definately better than Intel’s, but still not as good as NVIDIA. But at $4.4k AUD for 2 x R9700’s and 64GB of VRAM, i’m at 1/5th of the cost of a RTX6000 Pro with 95% of the benefit.
To polish my home system I have a cheap $20/mo Claude sub that i use to QA everything. The two Radeon’s can easily pay for themselves quickly if you are a builder and factor in a even a $200/mo Claude Max 5X sub, when you are building multiple ideas/projects daily over the course of a month will cap you out pretty quickly - so having two 9700’s that allows you to smash 80tok/sec+ 24x7 genuinely could enable you to build useful things.
So if you can afford it - my recommendation is to go Dual B70’s if you can afford the time to get stuff running - but if you want more plug and play experience, go the Dual 9700’s. It’s just easier
The irony is that, even with the recent pricing surge, a used RTX 3090 is still a better deal than a new B60 (as long as you only want one, given the cooling problems). For less money than the B60, they’re faster than the R9700.
Yeah, I contemplated getting a BattleMatrix setup, but decided against is because intel is so far behind the curve in terms of their software stack. I don’t see them fix it before the card before would become obsolete.
As for the 12VHPWR connections, I think you can get away with the conversion cables?
It’s an official Corsair cable that works with their v4 outputs. I checked an lots of people seem happy with them? It’s a 1000W psu, I’m not sure what else would be different. I’ll do so more searching on it.
But that is due to the 12VHPWR connector. I don’t see how a 12VHPWR adapter cable, as supplied with the card, is different from a the 12VHPWR cable supplied with the PSU in this regard, as they both share the same problematic connector?
I don’t really see the nvidia-style power connectors as a potential problem until you’re north of 400w through that connector. That connector style has been in use in servers for > 10 years and at least for the max 350w typical application there, I have only seen one melted connector in that scenario ever.
It can be a tough choice between B70 and R9700.. I am running fresh tests on tp=4 on both of them and they have different interesting problems. The r9700s seem faster and hvae wider software support, but if what you want to do is supported by intel’s llm-scaler that has been more consistently reliable over the longer term.
But I’m still doing lots of testing.
ps. dont sleep on the b60. 4x24gb is pretty awesome too.
It’s an interesting question, when you get to tp=4. Here in the UK you’re spending £4800 on the GPUs for R9700 or B70, plus some kind of system that can run them in 4 x PCIE 5.0 x16; that means Threadripper or Xeon, which is another £5k or so.
At that point, you’re not far off the price of a Blackwell RTX 6000 and any old PCIE 5.0 rig, which would smoke them both. Then it becomes a choice between (96GB + ridiculous performance + easy setup) and (128GB + reduced performance + setup headaches + software weirdness).
Thanks again for everyone’s input. I will start with the R9700 in the main x16 slot with no 6750 for now. No x8/x8 mobo yet, just the one card to start with. Will see how loud it is for gaming, and if adjustments need to be made. Will report back.
This card is definitely louder than the 6750. I don’t have a db meter to quantify it.
Honestly, I was expecting worse. I usually play with headphones and if I simply turn my headphones up a little, it’s not completely terrible. But CSGO is rather lightweight.
Trying out Cyberpunk now.
Oddly enough, it seems to be only pulling 9-10GB. How/why is CSGO consuming more VRAM than Cyberpunk?
Running rocm on a machine with a 6750 and an r9700 is not advised. Even a 9060 and an r9700 don’t use the same parameters in the setup (9070 and r9700 do).
