Intel B70 Launch - Unboxed and Tested

Oooft. If they stuck to the exchange rates, thus making the B70 around the £710 mark, it’d almost be worth a punt just to see if they can be made to behave themselves.

If they come in at near-R9700 prices, there’s just no chance.

1 Like

At the price point I would really like to see direct comparisons to a DGX Spark. 4x B70’s with 128gb total vram in a desktop config is fairly close in price to the Spark. Roughly same amount of memory (a bit more since you have separate system ram). So do apples to apples comparisons with all the models that will fit well on that setup (qwen 3.5, gpt-oss120b, nemotron 3 super 120b, etc.) I recall Alex Ziskind did a test on the Sparks where he ran it up to hundreds of concurrent requests and got massive token output, albeit I don’t recall the model he was using.

Ideally I’d like to see both single request, and then something like 50 concurrent and even 100+ concurrent requests, so I would know what kind of performance I could see if this were used as a dedicated offline OpenClaw Agent setup.

2 Likes

Hey there, I’m the OpenArc guy.

So CUMULATIVE_THROUGHPUT is from an older era in OpenVINO. In mid 2024 the partnership with huggingface began, introducing generative AI focus. Before then, OpenVINO was focused on computer vision tasks which have different inference requirements. OpenVINO documentation sadly never recovered from this madness, and tries to cover both. In that way, the deeper you go, the harder it is to tell what even applies to language models without studying the source code and experimenting like I have, to build OpenArc.

As I understand it, CUMULATIVE_THROUGHPUT requires application logic to work properly. But honestly, I’ve evaluated this and don’t reccomend it as a starting point.

If you want to do multi gpu with openvino here’s a snippet;

import openvino as ov
from openvino_genai import GenerationConfig, LLMPipeline
from transformers import AutoTokenizer

model_dir = ""

pipe = LLMPipeline(
    model_dir,
    device="HETERO:GPU.1,GPU.2",
    config={"MODEL_DISTRIBUTION_POLICY": "PIPELINE_PARALLEL"},
)

tokenizer = AutoTokenizer.from_pretrained(model_dir)

generation_config = GenerationConfig(
    max_new_tokens=24
)

prompt = "You're the fastest Llama this side of the equator. What's your favorite food? try to imagine"
messages = [{"role": "user", "content": prompt}]

prompt_token_ids = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="np"
)

num_iterations = 3
for i in range(num_iterations):
    print(f"\n{'='*50}")
    print(f"Generation {i+1}/{num_iterations}")
    print(f"{'='*50}")

    result = pipe.generate(ov.Tensor(prompt_token_ids), generation_config=generation_config)
    perf_metrics = result.perf_metrics

    print(f'Load time: {perf_metrics.get_load_time() / 1000:.2f} s')
    print(f'TTFT: {perf_metrics.get_ttft().mean / 1000:.2f} seconds')
    print(f'TPOT: {perf_metrics.get_tpot().mean:.2f} ms/token')
    print(f'Throughput: {perf_metrics.get_throughput().mean:.2f} tokens/s')
    print(f'Generate duration: {perf_metrics.get_generate_duration().mean / 1000:.2f} seconds')

    decoded = tokenizer.batch_decode(result.tokens, skip_special_tokens=True)
    print(f"Result: {decoded[0]}")

This code:

  • Runs a few generations
  • CORRECTLY configures the HETERO device plugin to split the model graph across specified devices
  • Uses AutoTokenizer, which was nontrivial to figure out works and is 100% an OpenArc fueled flex vs the pathway intel expects you to use, which sucks and is jinja hell.

–

OK so what is HETERO. As I understand it, Hetero distributes the model graph based on supported operations. At compile time, openvino runtime assesses device feature level and distributes transformer blocks based on kernel support for “nodes” described in the openvino_model.xml.

In OpenArc, you only configure this when registering a model once, and the readme is very easy to follow. Unfortunately HETERO device plugin is a dark art, not mentioned in the official release notes. I discovered support because I follow pull requests in all the intel repos to keep OpenArc up to date with features. Lately I’ve been working on hand optimizing models for openvino so we can support qwen3-asr and qwen3-tts, where I manually did what HETERO does automatically through the openvino compiler to get closer to real time tts with streaming.

I am an advanced OpenVINO user, and have been for almost two years now. I don’t actually reference the docs as much anymore. They are only updated on a quarterly basis when releases drop, vs tools like deepwiki, which reference the source code. Intel devs still hand write code, only adopting agents slowly, very recently. Look, I get how it sounds, but between OpenArc and the openvino notebooks, they are the place to start learning more of OpenVINO, NOT the documentation. It has problems, and I hate it. I’ve read it all, and spent hours fighting it, reconciling source code. There are so many things in OpenArc that aren’t in the documentation, and patterns you won’t see elsewhere.

However as a user, you dont need to worry about any of that biz if you are not interested. I have taken great pains to make this crap easier to work through in OpenArc, for myself and over time any one else interested.

one last note. OpenVINO covers many devices using a plugin system. This is source code speak; you don’t need to instally anything else. For example, recent qwen 3.5 linear attention has required new ops in CPU plugin and GPU plugin to leverage hardware features, and become integrated with the compiler. HETERO is a different, less documented animal.

ANYWAY, hmu with any questions, and try OpenArc if you want multi-gpu with openvino, or use the above snippet!

5 Likes

I relate to this deeply.

@wendell Whats with the slightly slower GDDR6 speeds vs. navi48 cards? Slower chips? Wonky memory controller? Both?

More to the point, when both products are aimed at AI usage, why on earth didn’t AMD and Intel at least opt for GDDR6X?

It would be nice to see the performance on llama.cpp SYCL, Vulkan and ipex-llm

probably not worth swapping 5090 for 3xb70?!

I might be wrong on this but I think GDDR6X is to some degree nonstandard. So more difficult to source at competitive pricing on the open market.

A better question, at least for Intel, is why in the world not GDDR7. Perhaps due to long release delay. This was supposed to launch a couple years ago, and maybe they’re stuck with an old design. It’s also curious there’s no AMD rdna4 refresh that uses GDDR7.

The main problem with Intel is their insanely long product release cadence. Battlemage was released in December 2024, but it’s taken them nearly 18 months to release the flagship card.

It makes no economic sense. They’re not capturing mindshare, they’re losing people’s interest. If the B70 came out even six months after the B580, it would’ve been a great product - instead, they let AMD beat them to the punch with the R9700, which means there’s almost no reason to put up with their shoddy track record on software with the B70.

I really want Intel GPUs to be good and do well, but they make it so damn hard to support them.

1 Like

I know it’s not the fastest card or the best supported, but it’s fun to tinker with, and good to support the third option in the market. I wish I could get one locally. It’s not the B770 I would have wanted, but it’s close enough.

@wendell Would it be possible for you to test Int4-AutoRound quantization on B70[1]? I think that the B70 lacks native FP4 support, but does support INT4. So with these new quantizers, Qwen3.5-27B would still fit onto a single B70 card (~20GB), run efficiently, and still have enough VRAM headroom for concurrent usage. At least that’s my hypothesis! :slight_smile:

In their paper on Arxiv, Intel claims that Int4-AutoRound is superior or at least equal to mixed precision 4-bit data types (Table 2 in the paper). The quantizer is open source, so you can create AutoRound models yourself, but for Qwen, Intel already provides HF models.

The llm-scalar README.md suggests that llm-scalar supports INT4 models. vllm serve Intel/Qwen3.5-27B-int4-AutoRound might be enough to pull and test this model. If that works, it would be great to see how a single B70 performs for batch processing.

If my math is mathing, 8 agents at 8k ctx should come in somewhere at around 100 tps aggregate, and 4 agents at 16k should hit 70 tps aggregate. I’d be very curious if that’s a correct estimate.

Thanks!

(Sorry, I can’t post links. Maybe my account is too fresh)

Oh, and one more thing, if I may: I’d be curious how Intel’s other (closer to mainline) vLLM builds perform. docker pull intel/vllm:0.17-xpu gets you the most recent one. llm-scalar to my knowledge builds on top of vLLM 0.14.0, so a much older vLLM version, but OTOH contains many more optimization patches.

With TurboQuant, hopefully coming soon to vLLM master, a more recent unoptimized vLLM version might, in the end, give you better throughput, because the improved concurrency may outweigh the worse single-session performance.

@wendell I appreciate these numbers. I have preordered a card myself. though in terms of the gaming numbers, do you know how comparable it is to the b580? as ideally this would replace my b580 in my home rig for my general testing and gaming purposes.

I don’t think anyone outside of Intel knows how the gaming performance will be given that there are no windows drivers at all for this yet.

well I am linux exclusive. so I dont much care about that.

2 Likes

If they could get the software side of their cards figured out I would certainly be a customer, though it is already arguably better than my cobbled together 3080ti and 5900x cpu offloading :smiley: .

@wendell when in idle, does the B70s fan go down to at least near inaudible?

i just bought a Radeon AI Pro R9700 from ASUS and the fan just won’t go below 30%. No tool, no sysfs call will change that. Now I’m considering a B70 but only if the fan is near silent in idle.

huge thanks, info would be REALLY much appreciated.

A comparison to the dgx spark, r9700 would be cool. Some other fine-tuning comparisons using the kappa model would be great for a follow up as the initial coverage seemed to focus on inference

2 Likes

So got my AsRock B70 in today. Had to swap it from my HPE DL385 Gen10 Plus V2 to a ROMED8-2T based system due to MMIO issues with the HPE. After install Proxmox 9.1.6 I was able to create 7 VFs (the listed max). What I found interesting though is it’s listing the card as a G21 instead of a G31.

root@no:~# lspci -vvv -s 0000:83:00.0
83:00.0 VGA compatible controller: Intel Corporation Battlemage G21 [Intel Graphics] (prog-if 00 [VGA controller])
Subsystem: ASRock Incorporation Device 6025
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin ? routed to IRQ 539
IOMMU group: 31
Region 0: Memory at 46e07000000 (64-bit, prefetchable) [size=16M]
Region 2: Memory at 45800000000 (64-bit, prefetchable) [size=32G]
Expansion ROM at ba200000 [disabled] [size=2M]
Capabilities: [40] Vendor Specific Information: Len=0c <?>
Capabilities: [70] Express (v2) Endpoint, IntMsgNum 0
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W TEE-IO-
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
AtomicOpsCtl: ReqEn-
IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-
10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Address: 00000000fee00000  Data: 0000
Masking: 00000000  Pending: 00000000
Capabilities: [d0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [110 v1] Null
Capabilities: [200 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable+, Smallest Translation Unit: 00
Capabilities: [420 v1] Physical Resizable BAR
BAR 2: current size: 32GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
Capabilities: [220 v1] Virtual Resizable BAR
BAR 2: current size: 8GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
Capabilities: [320 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration- 10BitTagReq+ IntMsgNum 0
IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy+ 10BitTagReq-
IOVSta: Migration-
Initial VFs: 7, Total VFs: 7, Number of VFs: 7, Function Dependency Link: 00
VF offset: 1, stride: 1, Device ID: e223
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000046e00000000 (64-bit, prefetchable)
Region 2: Memory at 0000046000000000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [400 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Kernel driver in use: xe
Kernel modules: xe




1 Like