4090 48g upgrade vs RTX A6000 (Ampere) in 2026

[wendell]

RTX 4090 (48 GB, Ada CC 8.9) vs RTX A6000 (48 GB, Ampere CC 8.6)

Both on PCIe Gen4 x8 — there is some TP=2 overhead trying to use both GPUs at once for TP=2 testing.

TP=1: 4090 dominates small models, A6000 wins on MoE

Model	4090	A6000	Winner
Llama-3.2-1B	321.2	225.4	4090 +42%
Llama-3.2-3B	131.5	90.4	4090 +45%
Llama-3.1-8B-Instruct	60.1	42.5	4090 +41%
DeepSeek-R1-Distill-Qwen-14B	32.8	23.3	4090 +41%
Qwen2.5-14B-Instruct	32.8	23.3	4090 +41%
gpt-oss-20B	118.5	117.8	≈ tie
Qwen3.6-27B-FP8	18.6		4090 only
Qwen3.6-35B-A3B-FP8	32.1	37.2	A6000 +16%

MTP on Qwen3.6-35B-A3B-FP8: Transformative

Watch the video; MTP has been a big boost to this hardware.

Config	4090	vs base	A6000	vs base
No MTP	32.1	—	37.2	—
MTP spec-tokens=1	55.2	+72%	61.2	+65%
MTP spec-tokens=2	77.4	+141%	85.0	+129%

TP=2: Mixed results — FP8 incompatible across gens

TODO: Explain this better

Model	TP=2	Note
gpt-oss-20B	94.7	Slower than single (PCIe bottleneck)
Llama-3.1-8B-Instruct	73.7	Faster (split compute helps)
Qwen3.6-27B-bf16	20.5	Only fits with TP=2 (52 GB)
Qwen3.6-35B-A3B-FP8		A6000 lacks fp8e4nv (Ampere vs Ada)

tool-eval-bench Quality Scores

This is nvidia’s tool eval bench

Model	Score	Notes
Qwen3.6-35B-A3B-FP8	97/100	Excellent — needs qwen3_coder parser
DeepSeek-R1-Distill-Qwen-14B	17/100	Never makes tool calls, just narrates
Llama-3.1-8B-Instruct	0/100	Makes calls but Hermes format mismatch

Mostly I just tested these two older models to show how far we’ve come for the video.

Visualizations

Notice here for models small enough to be tp=1 there is a performance regression for tp=2 – overhead of pcie to pcie communication. (This is why a lot of cards in the Ampere and even Ada generation had the bridges!)

• Throughput bar charts, TTFT/latency distributions, memory usage, FP8 comparisons, TP scaling, per-model summary cards

Key Takeaways

1. 4090 is faster for most models — 40-45% on dense models up to 14B. The 48 GB upgrade makes it a real A6000 competitor.

2. A6000 wins on Qwen3.6-35B-A3B-FP8 — +16% baseline, and 85 tok/s with MTP2 is the best overall result. Marlin FP8 kernel path on Ampere beats Triton FP8 on Ada for this MoE model. The speed is pretty close, though, just at more watts. Not enough to justify higher A6000 costs imho.

3. MTP is the biggest performance lever — 2.3-2.4× throughput on Qwen3.6-35B with zero extra memory cost. Probably? always use --spec-method mtp --spec-tokens 2 for this model.

4. Mixed-gen TP=2 has FP8 compatibility issues — Ampere (A6000) can’t run fp8e4nv kernels from Ada (4090). Non-FP8 models work but the overhead probably isn’t worth it. Maybe useful to have two for fine-tuning.

5. Qwen3.6-35B-A3B-FP8 is the best all-rounder — fastest tool-calling quality (97/100), good speed with MTP, and fits comfortably in 48 GB.

[someonetookmyname]

Awesome!

[jksoftware]

What about 3090’s? I’ve heard that is doable but my information could be wrong. Than I could get 2 3090’s with NVLink and 96gb of VRAM

[JordosTechShack]

Depends, your limited unless you change the entire PCB, or find rare 4GB chips. As far as I know you can only find 40 series replacement PCB with memory pads on the back side. He go overs what GPUs can be upgraded in the beginning of the video.

GPU upgrade. How its done ? Does it work ? What GPU can be upgraded ?

[Hero42]

Can you show the commands you used for benchmarking?

This is obviously a little different because I am using a quantization, but I was running unsloth/Qwen3.6-35B-A3B-GGUF:UD-IQ4_XS on a single 4090 24gb with llama.cpp and getting roughly 180 tok/sec. So I am curious how you are only getting around 32. Obviously FP8 is a larger model so there will be some performance difference, but I wouldn’t expect a 5-6x drop.

For context here is my setup.

docker run --gpus all -it \
    -v huggingface_volume:/root/.cache/huggingface \
    -p 8001:8001 \
    --rm \
    --entrypoint /app/llama-server \
    --pull always \
    ghcr.io/ggml-org/llama.cpp:full-cuda13 \
    -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-IQ4_XS \
    --n-gpu-layers 999 \
    --jinja \
    --ctx-size 262144 \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    -fa on \
    -t 8 \
    --parallel 1 \
    --host 0.0.0.0 \
    --port 8001

[jkyamog]

I gone to a different route for me to get 48GB of kinda RAM. Much easier in terms of hardware, just got a NVLink 3 slot for 2 3090s. But this meant I had to add software support for NVLink on lucebox.

https://github.com/Luce-Org/lucebox/pull/533

I do get almost 50% PP and 75% TG performance compared a 5090 running Qwen 27B (dense) on Q6 and 2x the context 128k vs 256k. These numbers are real world, 80-120k context.

On the first 48 hours of my usage I got a few M of useful work tokens out of them. Qwen 27B are my workhorses, they are fast enough and smart enough.

Curios to see if 48GB 2x 3090 NVLink vs 48GB 4090, can you run 27B Q6_K and get the numbers around 80-120k context?

[jkyamog]

Speaking of repurposing old enterprise hardware, this is what I use for Minimax M3 (added support on ik_llama.cpp) for V100s + Xeon (8 year old hardware)

https://github.com/ikawrakow/ik_llama.cpp/pull/1963

I am still in the middle of adding more V100s, been running with 2 V100s after I upgraded from a single 3060. Qwen 397B, Minimax M2.5 → M3.0, GLM 4.7 → 5.2. I am planning to complete at V100 4 board NVLink switch (6 direct channels). Only 2 GPUs are filled, waiting for more GPUs and heatsinks.

I will post sometime (hopefully soon) on my small YT channel… day job, family stuff, etc. YT is hard.

[eduncan911]

Saw this thread before the video and was, “Something doesn’t add up.” Now I have seen the video and holy hell!

Looking for the service now…

[0xKruzr]

Qwen3.6-27B-FP8 – 18tps on the 4090? you should be able to do a LOT better than this. have you tried Q8 in llama.cpp?

[aBav.Normie-Pleb]

• If I remember correctly the 3090 and 4090 were GPU pinout-compatible with each other.
• Early 3090 models had to use both sides of the PCB for VRAM chips to get to the 24 GB total due to the economical chip density at the time.
• Could you use the PCB of such an early 3090 to swap the GPU to a 4090 and up the VRAM to 48 GB?

Why?

• Basically to get an overbuilt model of a 4090/48 GB, not one of these “economically-designed” ones.

Modding a 5090, 32 GB to 48 GB?

Has anybody seen a mod of a 5090, giving it the denser memory chips as they are used on the RTX PRO 6000 to give it 48 GB, too - instead of the shitty default 32 GB?

[digitalscream]

There’s got to be something wrong with those Qwen 3.6 numbers, surely? I have no vLLM frame of reference, but the A6000 can run Qwen 3.6 35B A3B Q8_0 at over 100t/s in llama.cpp comfortably without MTP. Hell, a pair of 3090s in layer split can do about 140t/s.

Surely vLLM isn’t that kneecapped when it comes to the Qwen models?

[lambda]

I’m unsure about certain numbers as well. Looks like there is a bottleneck somewhere.

I can higher numbers on smaller quantizations on 7900XT via llama.cpp on certain models.

[vink]

Does anyone know if there has been any work done on getting P2P to work on the modded 4090? Unsure if getting 4 would be wise otherwise to run something like DSV4F compared to A6000s with two pairs of NVLink (but that’s a few thousand more expensive ).

[DOC_ZENITH]

I would like to know how does the undervolt work on this GPU?

Normally, on retail 4090 there is a lower limit at around 0,865, which usually gives you around 2,55Ghz on core while saving around 100-150W of power. In the past (till 3090TI, the OG 3090 was still fine) consumer cards could undervolt all the way to 0,7v, but then Nvidia applied some kind of lower limit for 3d undervolt, as in idle the card can still go lower. The PRO series of cards can still undervolt nicely, thats why then get the 6000 to 300W after all.

So my question is, how does this custom card undervolt? I looked around and saw only one video where the user was just lowering the power limit and thats not the optimal way to do it. Still, the clocks went down pretty fast, so I think they do not undervolt well at all. That would be sad as it greatly extends the GPU lifetime and also makes the card nice and quiet. 450W in 2-slot radial cooler is a jet engine. 300W is relatively quiet.

[HappyMedium]

Reviving this rather dead thread, in case anyone is interested about this rather nice piece of kit. I had a 4090 just past the end of warranty, so I took the plunge and got my card upgraded through the GPULab service. I also purchased the waterblock, taking Wendell’s hint that it might be a bit loud. Just wanted to drop my experience here.

1. The air cooled version is quite loud when running LLM inference. While not loud compared to datacenter hardware, if you have this thing running in a box in a normal household room and it starts ramping up, it sounds like a jet engine spooling up. The memory also gets rather hot. In a normal room temperature room, the VRAM got up to 88C which while it isn’t dangerously hot, probably isn’t ideal.
2. Because of the downsides of the air-cooled upgraded card, I decided to convert it over to being water cooled. The process is pretty straightforwards, the biggest challenge is actually pulling off the original heatsink, since the original thermal pads are quite sticky. I found it easiest to use a plastic spudger to slowly pry up each side of the PCB from several different points (starting at the end with the display outs). The PCB is designed nicely to have a rather large (~0.5-1cm) border zone around the edges that don’t contain SMDs. One tip which I only found later was that after placing the waterblock you need to torque in the screws quite slowly, methodically, but pretty firmly. I undertightened them the first time, and was getting really abnormally high VRAM thermals (in the high 90Cs when stress tested). Retightening until the screws down much more tightly (basically use a lot of hand strength, but don’t use your arm strength) was a bit nerve wracking but did bring VRAM thermals down to normal (80C max VRAM temp now when stress testing, GPU maxes out at around 64C, while Junction temp maxes out at 76C, which I feel like is pretty decent).
3. My homelab machine isn’t huge, it’s an Antec 1200 that I modded into a 9 HDD drive home server. Because of this, I really didn’t have space (or honestly the desire) to put together a full custom loop watercooling setup. So trying to get the setup as-close to a CLC I basically just attached the GPU to a Barrow 240mm DARIPD-30 combination radiator/pump/reservoir, actually attached to the outside of the case since I had insufficient room inside. I was actually expecting to need to power-limit the card to allow the 240mm rad to cool it, since most of the water cooling guides nowadays say a 240mm rad should only be able to cool 200 watts of heat. But oddly I’m able to let the card stay at its 450watt max TDP and the radiator is able to cool it perfectly fine (better than fine, I’ve been able to reduce fan speed down to 50% without any noticeable increase in temperatures, this is my first water cooling setup so I have no idea if this is something normal in people’s experience with water cooling or what) this is even when trying to heat-soak the water cooling loop (so running full tilt for like 1 hour).

Overall the water cooling version of the card is pretty awesome. 1-ish slots, running cool, very nice for the LLM curious home-labber. Yes, it’s very expensive (and I recognize and am very thankful for the extraordinarily privileged position I am in by already having a 4090 and the money to pay for it), but I also like that this service is based in the US and that I’m not paying Jensen Huang extortionate amounts for more VRAM for what is ultimately at the end of the day a hobby.