I’m trying to gather real-world information about Phison’s aiDAPTIV+ technology specifically whether it actually improves local LLM inference by extending GPU memory through their AI-enabled SSDs.
I’m looking for users who have hands-on experience with:
Phison aiDAPTIV+ SSDs
Any measurable improvements in LLM inference speed or model loading?
Does aiDAPTIV+ actually help run larger models locally by offloading memory to SSD?
What kind of latency or throughput impact did you observe?
Pascari AI100E or related Phison AI systems
Any real benchmarks or performance metrics?
Does it deliver meaningful acceleration?
Models from Newegg
I’ve seen aiDAPTIV+ SSDs listed on Newegg at around:
USD $2070 for 1 TB Pascari AI100E
USD $3350 for 2 TB Pascari AI100E
If anyone owns either of these:
Are these prices justified by performance?
Any benchmarks, latency measurements, or inference-focused metrics you can share?
Software requirements
Does aiDAPTIV+ require any special drivers, middleware, or Phison software to unlock the AI functionality?
Or does it work out of the box on standard Linux/Windows inference stacks?
Thanks in advance to anyone who can share real testing data or insight.
SSD wise, these are typical SCM (high endurance of 60-100 DWPD, low latency). They don’t perform significantly better or worse than competitors in the SCM space like Kioxia and DapuStor.
Regarding LLMs, these SSDs do not improve LLM performance on their own, nor do the SSDs themselves have any “AI” features. That is done with the bundled software, aiDAPTIV+ Pro Suite and aiDAPTIV+Link. For the moment, the software seems to be available “for free” and just requires a Phison Pascari AI100E drive as a sort of “physical license”, but who knows whether that will change. Link is just middleware itself, Pro Suite is the middleware plus a WebGUI as a central place for aiDAPTIV+ and model management.
The software acts as a middleware between the LLM and the GPU driver. Currently they only support Ubuntu 2024.04.3 LTS with Nvidia driver 550 (for 40 series cards) or 570 (for 50 series cards).
WIth the software in use the drives act as a cache, so the performance is worse than what you get from models that actually run in VRAM. The upside is that with a 2TB drive, you actually can load a 2TB model without having 2TB of VRAM.
It can also improve TTFT and token length, but with a reduced TPS as compared to VRAM only.
Source: I could play around with a client system at work for a week, running Llama2 70B and more on a GPU server with 2x 5090 blower cards and two AI100E 2TB U.2 drives. It does what it says on the box, but it’s not as simple as one might think. I have to be kind of vague and cannot post benchmarks due to a NDA with Phison. Make of that what you will.
Thank you very much for your detailed reply. I really appreciate you taking the time to explain how this works.
Just to phrase my question more concretely: today, for example, when I run MiniMax M2.1 quantized at Q4 (115GB), I get roughly 10–12 tokens per second. If I used the AI100E drive with aiDAPTIV+ and tried to run a model roughly twice the size, would you expect a major drop in throughput? What I am considering is potentially purchasing this drive mainly to see if I can realistically run somewhat larger models locally. Raw speed is not my primary concern. Of course performance matters, but I fully understand that for serious throughput I would either need to connect to my own cluster or pay significant costs for external compute. What I am really looking for is a practical middle ground.
I understand you cannot share exact numbers due to the NDA, but I would appreciate a high-level sense of what to expect. In your view, would it still be “usable” for interactive work, or does performance typically fall off enough that it stops being practical?
Also, do you know if there are plans to support AMD Air OCM Rocm with this software stack, or is support currently limited to Nvidia GPU only?