Help: Troubleshooting connecting to NFS share at 10G

Hi all,

I’m not sure if many here recall the stack I’m running but for purposes of this post:

  • UniFi 16XG switch(es)
  • pfSense primary router/firewall at ingress (with 10G NIC).
  • freenas.primary.intranet // Primary FreeNAS Storage with NFS shared. 10G IP is 10.0.2.2
  • Dell R740xd (64vcpu Dual Xeon Gold/256GB ECC RAM) // xcpng-host0. 10G IP is 10.0.0.152
  • XOA (from scripts) on vif 10.0.3.42

My VLAN setups is

  • SERVERS // 10.0.0.1
  • STORAGE // 10.0.2.1
  • VMS // 10.0.3.1

If I SSH into XOA, and run the following, I can easily write 40GB at 10G (well the equivalent that I’m seeing with MTU1500).

root@xcpng-host0-xoa:~# mount -t nfs -o vers=3,soft,proto=tcp,acdirmin=0,acdirmax=0 10.0.2.2:/mnt/big-primary/xcpng_data /mnt/xcpng_data

root@xcpng-host0-xoa:~# dd if=/dev/zero bs=4k conv=fdatasync of=/mnt/xcpng_data/dd_test count=10000000
10000000+0 records in
10000000+0 records out
40960000000 bytes (41 GB, 38 GiB) copied, 56.6226 s, 723 MB/s

I have tested iperf3 that yields 9Gbits/+.

However, if I SSH into the xcpng host directly, and try the above, on the first try the NFS share mounts, but any further attempts to list the directory etc, causes the command-line to hang up. Any ideas?

Saturating the interface? Could you try QoS?

I haven’t enabled QoS anywhere. Thanks.