Memblaze Pblaze7 7A40 Review 7.68tb

[wendell]

Background

The Memblaze PBlaze7 7A40 is a cutting-edge enterprise NVMe SSD designed to leverage the full potential of the PCIe 5.0 interface. Built on Memblaze’s Unified Framework Platform (MUFP), it delivers exceptional performance with 4K random read speeds reaching 3.3 million IOPS and 4K random write speeds up to 1 million IOPS—marking a significant milestone as the first in its class to achieve this level of write performance. Sequential read and write speeds are equally impressive at 14.1 GB/s and 11.2 GB/s, respectively. These advancements represent a substantial improvement over previous PCIe 4.0 SSDs, offering approximately 3x the random read and 2.5x the random write performance.

Latency optimization is a key aim for the PBlaze7 7A40, and in our tests it was very close to the claimed 55μs for 4K random reads. The 5μs latency for 4K random writes might be the best that I’ve seen on the test bench, but this required disabling PCIe ASPM (sleep).

The drive also boasts remarkable power efficiency, an improvement over past Memblaze drives, but it manages to stay inside a 22w power envelope in our tests. For read-heavy workloads, the power usage peaked at just 13.33W while writes averaged closer to 18w. This would be pretty useful for dense deployments / 24 drives mounted in the front of a 2u chassis.

In addition to performance, the PBlaze7 7A40 supports NVMe 2.0 and NVMe-MI 1.2b protocols, offering enterprise-grade features such as full data path protection, power-loss data protection, and firmware upgrades without the need for system resets. I was also able to configure multiple namespaces.

The drive is implemented with a PCIE 5.0 Gen5 interface, and 3D TLC nand.

It has an advertised endurance/warranty of up to 1 drive write per day, for 5 years.

Learn More.

Benchmarks

Reads (with filesystem overhead / ext4 )

Job	Read MB/s	Read IOPS	Read Avg Lat (µs)	Read p99 Lat (µs)
seq_read_128k	12729.95	101840	1256.70	2768.90
seq_read_1m	13562.10	13562	9436.57	11206.66
rand_read_4k	11568.30	2961484	172.71	626.69
rand_read_128k	10256.66	82053	6238.99	20840.45

Without filesystem: 3297521 IOPS (Random 4k) / 14019.1 MB/s / 56.88 us

Writes (with filesystem overhead / ext4 )

Job	Write MB/s	Write IOPS	Write Avg Lat (µs)	Write p99 Lat (µs)
seq_write_128k	10737.76	85902	1489.90	1662.98
seq_write_1m	10711.33	10711	11949.25	12910.59
rand_write_4k	10723.18	2745135*	186.28	329.73
rand_read_128k	1117.16	8937	57284.51	65273.86

Without filesystem: 591411 IOPS (Random 4k) / 10795.33 MB/s / 5.61 uS
*buffering

Workload Testing

TO PASTE

Database

TO PASTE

S3-style object storage

TO PASTE

AI Model load time

This is effectlvely just ollama load (blep) from an ext4 volume.

TO PASTE

Conclusion

This drive performs well, has a nice heatsink design, and goes toe to toe with our previous best-in-class the Kioxia CM7. I was impressed by this drive. Memblaze has been around 13 years and they are doing more marketing to make their accomplishments known, time will tell how this holds up.

Test Platform

AMD EPYC 4585PX 16-Core Processor
CPU-attached PCIe Gen5

[jode]

I guess it’s one of these “if you have to ask for the price you can’t afford it”

[BillDStrong]

The 3TB version I found on Alibaba is 900 US, so yeah.

[Exard3k]

@jode

I guess it’s one of these “if you have to ask for the price you can’t afford it”

Those numbers on the writes are getting bigger and bigger with every generation. And raw disk 590k IOPS on 4k with <10 microseconds latency…that’s beyond the scale really.

NVMe controllers are getting more scary every day.

Do we need that? I certainly don’t have a use for it. Storage hardware isn’t my bottleneck. But I know the innovation pipeline for future upgrades is working. So I can look forward to just buy a drop-in replacement in a couple of years.

And new drives in 2025 are the ebay/EOL/budget drives of 2029 after all. I’d rather see cheaper NAND than faster controllers though

[lemma]

@BillDStrong

The 3TB version I found on Alibaba is 900 US, so yeah.

I’m having no luck. There is one refurb Pblaze7 7930 on eBay.

@Exard3k

Do we need that? I certainly don’t have a use for it.

Yeah, same. I have too many other job responsibilities to have much time to write code that can take advantage of the modded SM2508 in the SN8100. And diminishing returns mean the overall workflow benefit for paying US$ 140/TB instead of US$ 75/TB for SN850X borders on negligible.

From an IO enthusiast perspective, sure, I’d like to have an SN8100 to develop code against as it does things a P5800X can’t. But there’s nothing there that’ll suffer by waiting a year for more reasonable prices.

[Exard3k]

@lemma

And diminishing returns mean the overall workflow benefit for paying US$ 140/TB instead of US$ 75/TB for SN850X borders on negligible.

I don’t wait on storage hardware these days. I wait for CPU, code, GPU, more code and networking almost all of the time. NVMe just erased that from the equation for me. I certainly see some saturation effect and the benefit curve slowly flatlining for every new generation of NVMe controller.

And for my server…I’m into Ceph and used ZFS before, both are awesome. But the limiting factor now is CPU, memory bandwidth and even stuff like congestion in the Kernel and other low-level stuff that wasn’t built for that bandwidth and IOPS and is often 10-20 year old code written for an age that has passed.

E.g.: If you sequentially read/write 1GB/s from a node in Ceph, that node uses 6-10GB/s in memory bandwidth to do so. Given, that’s a rather complex and abstract storage system and not your average SMB file copy. But that’s an enterprise product of RedHat/IBM and people there talk about IOPS/core, not IOPS/drive. And even 12x DDR5 channels won’t do the job in processing PCIe5 NVMe bandwidth or IOPS for this kind of storage without bottlenecks.

Stuff in general needs to catch up with revolutionary NVMe storage

[lemma]

@Exard3k

Given, that’s a rather complex and abstract storage system and not your average SMB file copy.

Even the latter’s easy to run at 6x the DDR bandwidth as the copy. With buffered IO there’s read from disk to OS buffer, copy to app buffer, the app read, app write to app buffer, copy to OS buffer, and write from buffer to disk. So one PCIe 4.0 x4 drive can saturate dual channel DDR4.

Even just doing read I often have to write unbuffered IO to free up enough DDR4 bandwidth that workloads can get to PCIe 4.0 x4 drive potential. Expecting that’ll happen again with DDR5 as soon as I move 5.0 x4 drives, though not as much since it’s easier to run out of cores first.

@Exard3k

even stuff like congestion in the Kernel and other low-level stuff that wasn’t built for that

Yup. 7 GB/s is 7000 1MB reads/s. Asynchronous IO isn’t built for that, so I find it benches more efficient to do synchronous IO and just block threads until the drive returns. Increase the buffer size to try to get into IO completion ports’ speed range and the L3 evictions hammer DDR. Plus async IO isn’t as DDR efficient as sync on the OSes I’ve tested on.

That’s one reason I wanted to make my last build 9950X3D. They were all out of stock, though.

@Exard3k

Stuff in general needs to catch up with revolutionary NVMe storage

Mmm hmm. I’m finding I’m basically doing programming research, and not in the colloquial sense of looking stuff up on the internet. Some of the dependencies I’m stuck with really don’t want to go above 3-6 GB/s and some of them are closed source, so trying to patch them for better performance isn’t happening.

In my own code I’ve a couple ideas to try to get more stable performance characteristics. Not sure when there’ll be time to code and bench them, though.

[Exard3k]

@lemma

so I find it benches more efficient to do synchronous IO and just block threads until the drive returns

At this point I just turn into the devil’s advocate and be an Enterprise NVMe salesman and sell you some nice drive (like the one in this thread) with PLP capacitors where the drive just reports back basically “just like it was Async, the capacitors will ensure it’s on stable storage”. And suddenly you’re dealing with several hundred k of IOPS of synchronous IO per drive and 14k x 1M reads or queues of 128+ if you want to feed the drives so they don’t run dry.

But this also complicates your task and time to take a deep breath. But your SN8100 certainly saved you from all those engineering challenges

I certainly don’t envy software engineers trying to make things work and reinvent stuff for rockets that were made for wheels, with this kind of hardware.

@lemma

That’s one reason I wanted to make my last build 9950X3D

For the L3 and the bandwidth? SRAM Certainly interesting stuff to dig into. Same as HBM. Good old SDRAM certainly hits more limitations with every year passing.

[lemma]

@Exard3k

async

Not that async. Seems like you’re thinking of zfs’s overloading of the term to refer to write through rather than the threading model used in IO calls.

So, no that’s not what Sandisk’s modded SM2508 in the SN8100 accomplishes. It does somewhat exceed P5800X in IOPS while capable of similar latency figures and also being about as good as it gets for 5x4 sequential. Fits in 2280 desktop M.2s and takes normal NGFF heatsinks too, just needs to drop off the enterprisey $/TB launch pricing to something more appropriate to 0.33 DWPD before it it’s likely to make much sense.

There are hints of inconsistent throughput in some SN8100 reviews, so I’m also kinda waiting on the write until drive fills testing Tom’s Hardware does. If there is a problem and Sandisk does a firmware update before I get around to buying one that’s fine too.

@Exard3k

Same as HBM.

You could make similar case around dGPU’s wide GDDR and Apple’s wide DDR interfaces I think, but I don’t see something like that happening anytime soon in the AMD and Intel sockets I primarily code for.

It’s not really a big deal. For synchronous IO, where the calling thread blocks until requested data returns from or the kernel finishes sending data to the drive (rather than asynchronous IO, where the thread does not block and data completion invokes an app code callback), you just keep enough threads in IO calls to fully utilize the drive. Basically, instead of SEQ1M Q8T1 it’s SEQ1M Q1T8.

The fiddly part is scheduling the optimal number of threads is different for every drive, workload, OS, and potentially core performance order. My experience with 4x4 is utilizing ~80% of a drive’s potential bandwidth’s pretty easy and reliably getting the last 20% is pretty involved. Like my code’ll do 100% no problem but after a few seconds Windows 11 starts scheduling differently and it drops to 80%.

[lemma]

@lemma

I’m also kinda waiting on the write until drive fills testing Tom’s Hardware does

3.9 GB/s TLC direct and then bouncy folding between 1 and 3 GB/s that probably averages close to 2 GB/s. So starts to lag some of the competition for writes past 430 GB/TB capacity.