checking file integrity as the file is being moved over the network.
It’s being read from my RAID0 array consisting of four HGST 6 TB 7200 rpms HDDs, which are capable of a total of somewhere around 750 MB/s read speed for the entire array. (~187.5 MB/s per drive based on iotop).
Currently, best case scenario (using an Intel Core i7-3930K, a single thread can only read at around 311 MB/s with 64 GB of DDR3-1600 memory I think, I forget the timings).
But if I am reading the file over the network, I’m reading it over 4x EDR 100 Gbps Infiniband (which has a theorectical peak bandwidth of 12.5 GB/s, which I should be able to obtain if I move to U.2 NVMe SSDs).
But at the current rate, I think that I am definitely CPU bound.
When I run
openssl speed sha256, I get around 350 MB/s for 8 kiB block sizes.