FreeNAS transfer speed issues

Hello so after a few problems with an HDD I had I have managed to successfully setup (at least the first part of it) my very first FreeNAS machine. However I have been having some major speed problems. Regarding the hardware I should be more than good enough:

Mobo: Super micro X8SIE
Ram: 16gb ecc ram
cpu: intel x3430 xeon
hdd: 5 x TOSHIBA MD04ACA500 5TB 7200 RPM 64MB + 1 x TOSHIBA X300 HDWE150XZSTA 5TB 7200 RPM 128MB (RAID Z2 ~18.2 TB)
PSU: Seasonic 550w
network switch: NETGEAR 8-Port Gigabit Ethernet Switch (GS208)

So my problem is that whenever I am transferring files I usually get about 2-3 MB/s and occasionally 11-12 MB/s which are both way too slow. I performed a speed test on the shell and the speed I received there was about 50MB/s for the entire array. However when I tested the WRITE speed of individual drives I was getting above 90 MB/s. I have found from another post the following "setting/command" socket options = TCPNODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536_ but I do not know how to use it (yes I tried googling it). That command should take care of my problem regarding the 50MB/s array speed, but my main problem is the network speed that I am getting.

It appears that the speed is capped to 100Mb/s ~ 12MB/s which does not make sense, since both my main computer and the NAS have 1000T-Base intel ports, the switch supports 1000Mb/s and I have used only cat5e cables for very short runs.

Anybody that can help I REALLY appreciate it.
Thanks in advance!

That is meant for SMB, so you go to services - SMB - and there at Auxiliary parameters you add
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536 and hope that it helps.

Thats said I am assuming you use SMB. Have you tried some other service as well as usually its SMB from Linux to Linux that sucks that badly

1 Like

Thanks for the reply, unfortunately even though I did exactly what you told me. Whenever I am doing a speed test of the entire array I am still getting about 50MB/s instead of 100 or a bit over. Also my main problem, which is the network transfer rate is still extremely slow for some unknown reason. I am getting a maximum of 12MB/s even though all the ports and the switch and everything are rated as 1Gb/s cabable which would be equivalent to about 120MB/s.

I'd say that because you're getting 100mbps max that somewhere between the two devices there is a 100mbps link. Might be worth checking your cables and all that.

How are you testing the speed of the array?

All the cables I am using are short runs (less than 10ft) of cat5e which should be more than good enough to get 1Gb/s or 120MB/s. Also both the switch and the router are gigabit, even though the traffic is not going through the router to go to my computer. To test the array I am using the following command :

dd if=/dev/urandom of=/tmp/test.rnd bs=1M count=1024

and I get the following output:

1024+0 records in
1024+0 records out
1073741824 bytes transferred in 21.683739 secs (49518297 bytes/sec)

Which is about 50 MB/s. However whenever I am transferring files between the NAS and the main computer through a wired connection I am getting a maximum of 12MB/s which is 100Mb/s speeds not gigabit. Also the command I have posted above is supposed to make the array faster to about 100 or 150 MB/s but even after a few reboots it does not have any effect on the speed. When I test each drive individually I get a minimum of 90 MB/s on the innermost side and about 180 MB/s on the outer side. So there is potential I just do not know how to extract it and use it. Also my cpu should be more than good enough since its a quad core intel and it never gets above 10% usage in FreeNAS

Using urandom is probably the bottleneck, try creating a file with urandom and then moving it on to the array after creating it. Also do you have deduplication enabled? As that will be a performance hit especially with random data.

If one of your cables is damaged, it may only work at 100mbps as gigabit needs all eight wires instead of just 4.

1 Like

Damn thanks a lot I never thought to check the cables after installing the system. It turns out one of them was pretty badly damaged. After replacing it I am getting speeds of about 100-110 MB/s which is pretty good for gigabit. Now I have a last problem which I do not know what it could be causing it.

Whenever I am making a transfer of files to a windows share on the freenas if I am transferring more than a few gigabytes, the system will halt, stop the copying and unless I reboot the machine, it will be stuck in a loop trying to write the files. I have no idea what it could be causing this issue. Is there anything I can provide here to help in diagnosing the problem?

Thanks in advance.

1 Like

What's your memory usage like? If you have deduplication enabled with that much storage then you will need way more ram to handle it.

Right now the deduplication is set to inherit(off) by default. As for RAM usage I am having more than 14 GB free so I should be fine. Here is a screenshot from my ram usage:

I took a picture a few minutes ago from the FreeNAS console when it was about to crash:

Anything in the system log?

If you mean in the alert messages in the upper right corner no there is nothing there and the light is green. Other than that I do not know where to look

I've never used freenas so i'm not sure either, but it should have some sort of system log

Ok I finally found where the messages are displayed I am pasting the last few minutes before and after the crash:

Nov 1 08:53:44 freenas smbd: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mdnsd Socket:35 Err:-1 Errno:2 No such file or directory
Nov 1 08:53:48 freenas mDNSResponder: mDNSResponder (Engineering Build) (Oct 3 2016 06:58:50) starting
Nov 1 08:53:48 freenas mDNSResponder: 8: Listening for incoming Unix Domain Socket client requests
Nov 1 08:53:48 freenas mDNSResponder: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
Nov 1 08:53:48 freenas mDNSResponder: CheckNATMappings: Failed to allocate port 5350 UDP multicast socket for PCP & NAT-PMP announcements
Nov 1 08:53:55 freenas mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000801406D60 freenas.local. (Addr) that's already in the list
Nov 1 08:53:55 freenas mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000801407180 135.1.168.192.in-addr.arpa. (PTR) that's already in the list
Nov 1 08:56:18 freenas notifier: Stopping ntpd.
Nov 1 08:56:18 freenas ntpd[1964]: ntpd exiting on signal 15 (Terminated)
Nov 1 08:56:18 freenas notifier: Waiting for PIDS: 1964.
Nov 1 08:56:18 freenas notifier: Starting ntpd.
Nov 1 08:56:18 freenas ntpd[6895]: ntpd 4.2.8p8-a (1): Starting
Nov 1 08:56:19 freenas notifier: Performing sanity check on nginx configuration:
Nov 1 08:56:19 freenas notifier: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
Nov 1 08:56:19 freenas notifier: nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Nov 1 08:56:19 freenas notifier: Stopping nginx.
Nov 1 08:56:19 freenas notifier: Waiting for PIDS: 2785.
Nov 1 08:56:19 freenas notifier: Performing sanity check on nginx configuration:
Nov 1 08:56:19 freenas notifier: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
Nov 1 08:56:19 freenas notifier: nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Nov 1 08:56:19 freenas notifier: Starting nginx.
Nov 1 08:56:19 freenas notifier: kill: 3134: No such process
Nov 1 08:56:19 freenas notifier: Performing sanity check on nginx configuration:
Nov 1 08:56:19 freenas notifier: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
Nov 1 08:56:19 freenas notifier: nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Nov 1 08:56:19 freenas notifier: Stopping nginx.
Nov 1 08:56:19 freenas notifier: Waiting for PIDS: 7030.
Nov 1 08:56:19 freenas notifier: Performing sanity check on nginx configuration:
Nov 1 08:56:19 freenas notifier: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
Nov 1 08:56:19 freenas notifier: nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Nov 1 08:56:19 freenas notifier: Starting nginx.
Nov 1 08:56:20 freenas mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000801406D60 freenas.local. (Addr) that's already in the list
Nov 1 08:56:20 freenas mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000801407180 135.1.168.192.in-addr.arpa. (PTR) that's already in the list

Well in this output there is nothing regarding the acutual file serving/shares.
I am not at home, but when I am I will dig into my own box to see where the logfiles are that might help us track your bug down.

Btw. glad you found the faulty cable... they are a pain, a real pain when trouble shooting. Even more so when they look happy and fine from the outside, but have broken wires inside.

Thanks for all the help, since the problem is different than this topic I will be opening a new one, to have a more accurate title and hopefully find a solution.

Cheers