could you post your tunable numbers maybe something with mine is just off
The retries are unexpected. Can you try a different cable? Also I’m curious the output of netstat -m and netstat -s all (as text attachments, preferably). Those might be specific to a particular jail though, I’m not sure. Might take us a few tries to get the right stats.
Ya i can try 3 more cables. lol. I’ll have to redo that later tonight. have some yard work to do today.
i attached a txt doc of both netstat -m and -s
Netstat data.txt (38.8 KB)
There is a high number of TCP fragments discarded “due to memory problems”. This is actually not a very good description of that counter. It actually counts when a packet is discarded because the reassembly queue has reached a max length. This defaults to a low value to prevent DoS attacks, but you can raise it to effectively eliminate the limit.
Try tuning your TCP reassembly queue max len to a higher value:
net.inet.tcp.reass.maxqueuelen=1437
1 Like
Turned into a crazy weekend so didn’t have time to test the other cables.
I’m running 64GB ecc ram so i wouldn’t think id be short. I have jumbo frames turned on on my switch. Not really sure how to turn them on on the FreeNAS box. I don’t think i have MTU running but not sure.
Well RAM amount isn’t a problem, but a memtest may be good to run on it. It’s possible you may have a DIMM acting up.
Though a new cable and verifying MTU sizes are unanimous on both NICs would be a faster test, so I’d do that first
I’ll have to look into the MTU test tonight and see how to do it.
They are brand new dim’s but it is possible. I should have time tonight to check a new cable and run tests.
I’m starting to wonder if i should toss the drives, 10 gig card and flash drive into Tower that has the AMD 8350 rig to see if i get full speed. Then i’ll know its not a card or freenas issue just a server issue. Then i’d just have to Find a board that would support the cpu’s and ram.
EDIT: see my previous post about the “memory problems” label being misleading.
(somewhat irrelevant link from before my edit, I’ll leave for posterity)
I’m running FreeNAS-11.1-U6. Does that mean i should look into this?
Freqlabs you’ve been a huge help I just want to make sure I’m testing the correct things.
FreeNAS 11.x is based on FreeBSD 11 so these things do apply, yes. It probably only matters if you are using Jumbo Frames though.
I don’t think i am. I havn’t looked into how to turn that on or off.
I don’t think a memory test will reveal anything.
If you can post your ifconfigs here as well, that would answer a few questions for me. Sorry I don’t feel like joining the FreeNAS forum right now.
If it turns out to be a FreeNAS issue you should definitely file a ticket in their Redmine. I have seen some recent work on the ix driver in FreeBSD as well, that might need to be merged into FreeNAS before the 11.2 release (or in a later update if the timing is not right). I’m not sure if it is applicable or related to this issue off the top of my head though.
For now I’ll wait until I can take a look at your ifconfig before considering what can be tested to explain the allocation failures.
heading to my house now would a text file be better than pics
I just realized I mixed something up so I’m about to edit a post, stand by 
not a problem
So you’ll never believe what just happened.
i set the mtu to 9000 and i am now getting 11.1 GBytes / 1.11GBytes/sec in iperf, I am going to test with a 5 .6GB file right now.
1 Like
With the setting at 9000 i am getting full transfer speeds.
server says
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 9.27 GBytes 7.95 Gbits/sec
I guess my network just didn’t like the default MTU 1500
Having now realized I mixed up two different error counters, I do actually believe this 
Try also making the tunable change mentioned above. Then clear your counters, run the test again, and measure the errors. I’m also curious about the loopback test in the jail after these changes. It would be good to compare before and after both changes separately and together, if you can!
# remove ix0 from the bridge!
ifconfig bridge0 deletem ix0
# reset counters
netstat -p tcp -s -z
# ... run your test ...
# get new stats
netstat -p tcp -s
# put ix0 back in the bridge
ifconfig bridge0 addm ix0