Kioxia CD8-R PCIe 8x AIC (Gen 4)

Before buying anything else check the mentioned PCIe AER settings.

Since you’re running an EPYC CPU and are using PCIe lanes coming directly from the CPU PCIe AER should work properly to help you diagnose stuff (on AMD systems PCIe AER doesn’t work with PCIe lanes coming from the Chipset instead of the CPU).

PCIe Adapters can introduce issues so that the PCIe Error Correction has to step in and repeat transfers (basically the same as lost packets when using a network connection with issues) which results in lower usable performance.

But it can also be that the SSD’s firmware isn’t optimized for this kind of sequential workloads but only reaches maximum throughput when doing many transfers in parallel.

If you don’t have any PCIe Bus Errors in your log I’d boot Windows and do a benchmark of the drives with CrystalDiskMark to have a “clean” reference point.

Another user had a similar issue with enterprise gear and it turned out to be some suboptimal configuration in Linux, not a PCIe Adapter issue and the SSDs were completely fine under Windows.

PCIe NVMe HBAs open a whole other can of worms (refer to my other thread that evolved over the years and is still ongoing: A Neverending Story: PCIe 3.0/4.0/5.0 Bifurcation, Adapters, Switches, HBAs, Cables, NVMe Backplanes, Risers & Extensions - The Good, the Bad & the Ugly ).

Using cables is much harder for signal integrity compared to a quality PCB.

3 Likes