Has anyone else had issues with tensor parallelism recently breaking on arc b70?

Hardware Hub
, ,
1

I have two Intel Arc B70s. I custom-compiled the Intel SYCL stack and llama.cpp.

I had tensor parallelism working before, but sometime in the last two weeks something in the Intel stack broke. Now I’m getting memory exceptions that seem to be coming from the xe driver when running a model in tensor parallelism.

Layer split still works fine, even with very large contexts, but it’s a lot slower than tensor parallelism was.

Anyone else seeing this? Trying to figure out if it’s a driver/kernel update or a regression in the SYCL stack.

1 Like
2

how did this end up working out for you?

3

You need to roll back the Intel SYCL driver to the stable version from two weeks ago and check the compatibility of the libze_loader library.

4

it works ok. not the fastest but it can run full Q8 models with large contexts.