I have a setup where I keep my OS and all my files on a Optane P1600X in a Sabrent Thunderbolt enclosure and use this as a portable boot device. This let’s me use my more powerful workstation when I’m at my main desk and my laptop when I’m on the go (previously documented here). This has it’s obvious drawbacks, but for my use it works surprisingly well!
I’ve have however been curious as to what kind of performance hit I incur by using the drive via Thunderbolt instead of direct NVME, so tonight I did a quick test of my various setups and below are the 4k/T1/Q1 read results:
I didn’t try connecting the drive directly via NVME/PCI-e, but from Googling I see that the P1600X maxes out at about 95k IOPS at 4k/T1/Q1, so overall I’m pretty happy with these results!
For reference I also tested to see how close I could get to the drives max speeds from the Intel data sheet and I managed 350k IOPS at 4k/T1/Q32 and 1475 MB/s at 128k/T1/Q32 over Thunderbolt. Intel states 410k IOPS and 1760 MB/s. Note that the Sabrent enclosure only has 2 PCI-e 3.0 lanes, and not the full 4 that the drive supports, so this might affect things.
Latency wise on Thunderbolt I get the following figures for 4k/T1/Q1:
Best case (drive → desktop): lat (usec): min=12, max=306, avg=13.43, stdev=1.86.
Worst case (drive → docking → laptop): lat (usec): min=19, max=1397, avg=29.45, stdev=18.6
The data sheet states 7µs latency for 4k reads, so I’m getting about double that in the best case or four times that in the worst case. Still not too bad though, considering that a Samsung 990 Pro has about 130µs of latency for the same use case…
I now mounted the same Optane P1600X in a M.2 slot on my motherboard and measured performance and latency for 4k/T1/Q1 there and got the following results:
Performance: 440MB/s, 107k IOPS
Latency: lat (usec): min=8, max=3517, avg= 8.86, stdev= 5.27
