Razer Core X + 4070 Ti Super Performance for Local AI Workloads

I’m planning to use an MSI RTX 4070 Ti SUPER EXPERT (16 GB) inside a Razer Core X with an LG Gram laptop over Thunderbolt 4 for local AI work (RAG pipelines, embeddings, local LLM inference, and possibly small-model fine-tuning).

Does anyone here have real-world experience with:

  • 4070 Ti Super in a Core X

  • TB3/TB4 bandwidth limitations for local LLMs

  • Ollama / LM Studio / llama.cpp / QLoRA over eGPU

  • thermal stability during long runs

  • performance loss compared to desktop PCIe

Also, does using an external monitor connected directly to the eGPU noticeably help for AI workloads?

Thanks!

MoE models an anything that needs to go back and forth on the USB4 bus will suffer. Any “monolithic” model you load in memory will be just as fast as it would be on a desktop PC.

In short:

  • Don’t. Sell the Razor Core X.
  • Buy another 4070 Ti, whatever it takes, to have a total of 32GB of VRAM.
  • Buy a motherboard with dual x8 slots to the CPU, like ASRock X570 or X670 or X870, etc.
  • 32GB of DDR4 is fine.

/TL;DR

NOTE: Razor Core X is Gen3 and TB3 x4. Super super slow.

There’s a few things here to define… LLM and Generation. LLMs will generally run great, regardless of the PCIe/TB bus.*. Image/video/re-training will suffer greatly over slow PCIe/TB links.

*With LLMs, there are two measurements:

  • Prompt Processing (PP), or how fast it can read your system and user messages, as well as prompts, and uploads (PDF, images, videos - per frame analysis!, Doc or even 500,000 lines of code, etc).
  • Token Generation (TG), or how fast inferencing is measured. This is the streaming text you see for your answer.

PP will vastly suffer over slow PCIe or any TB connection (they are only x2 and x4 lanes). Have a 500,000 line codebase to reference? Yeah, all that context had to be loaded every time you submit a prompt. A slow connection means very slow movement from CPU to GPU VRAM (e.g. xferinf a 200MB PDF). On a slow system, we are talking several minutes to compute everything - before it starts to think and use tools.

TG is about the only think that’s not really affected here. Inference is nice and fast - if you can fit the entire model into VRAM. It’s just the initial PP will be much slower.

Also model changing will be super slow (mine over Gen5 x16 is about 2-4 seconds. On my PCIe4 system, it’s about 6 seconds to switch.)

Also, from my Razor Core X on Linux over TB3 years ago, it was a bit unstable. It would either show on boot… or, not. And it required many complete power off cycles to get it to show again. Firmware updated and all.

I personally would not waste my time with eGPU enclosures and Ai.


With all that said, you want to stay far away from ANY type of CPU offloading. You will have to use models that 100% fit in 16GB VRAM.

However, right now - in the last 4 months - new models have gotten so good to actually rival frontier models! The problem is, that threshold is usually 24GB right now - and you really need 32GB of VRAM to have enough room for models and context and cached processing to make it viable.

I speak from my 16GB VRAM laptop experience. Doh.

So if all you got is the 4070’s 16GB of VRAM, IMO I would buy a second 4070 Ti to match. Next, I’d find an ASRock X570 motherboard (Gen4, AM4 socket) as their X570 splits the Gen4 x16 into dual x8 Gen4. Most mobos don’t do that any more. That’s the lowest cost to entry, while gaining a full 32GB of Gen4 x8 goodness that will smoke!

But maybe, in another year, we’ll have higher performing and small models? The trend sure does seem that moving that way. :slight_smile: