Ultra low latency filesystem/folder network share, need an advice

Networking Networking Software
1

TL;DR
Having a optane 905p and try to share it with another PC, what hardware should i buy and what software/architecture to get a ultra low latency setup. Low latency is more important to me than raw throughput (1250MB is more than fine, even less)

Okay, why i need such thing?
Iam a developer and most of my work (during development) is running in docker-containers with file mounts. Spinning up multiple containers at the same time (microservices) sucks d. on a local pc (already have a TR3960x). Sometimes multiple containers also need the same mount-points (Iam a freelancer and this setup is pretty common).
Fast prototyping, so my IDE (PHPStorm) and the containers having the changes almost immediately is key.

Options (software):

  1. Fine-Tuned NFS
  2. Some cluster FS like ocfs2 / lustre
  3. Something i don’t know yet

Options (Connection point2point):

  1. 10/40gbe network - what nic?
  2. Fibre channel 16?
  3. Usb/others?

Goal:
Having a “identical” container running locally and remote with one filemount.

Price goal <1000 EUR

Thx for your help!

Cheers
Alex

2

Put an SFP+ NIC in both machines and use a Direct Attach Cable between them. Should give you lowest possible network latency. 40Gig SFP might help, cost is a bit steep though.

BeeGFS :stuck_out_tongue:

image
image1199Ă—888 100 KB

3

Whatever remote filesystem you end up using, the secret code for low latency is RDMA: Remote Direct Memory Access.

Instead of doing normal networking, this allows one system to tell both network cards involved to copy memory from one system to the other. All the details of TCP, IP, Ethernet are abstracted away and the CPU is only worried about the results.

Even though I mostly use Linux myself I have only seen this done with Windows SMB Direct.

I did find this just now but I don’t know how up to date it is:
https://www.kernel.org/doc/Documentation/filesystems/nfs/nfs-rdma.txt

3 Likes
4

Hi @MazeFrame @zlynx ,

thx for your answers, i’ll try to find a decent 40gbe rdma card and hope for the best with linux.

Cheers
Alex