Here’s the results from testing FreeNAS stock, with tunables, with mtu 9000, and with both on my RAIDZ2 8x4TB single pool. It’s a VM to VM test, so the networking stack is virtualized, providing a best case scenario
The biggest gains are gotten from setting Jumbo Frames (MTU 9000) throughout the network path. The tunables can increase performance mostly on the Read section, but also give a minor bump to writes
Overall I don’t think the majority of those tunables are helpful or necessary. Also, there is an autotune built in to freenas somwhere that you might try instead.
I’ve enabled that before. The autotune works pretty alright, but I also didn’t find most tunables to really matter much. My NAS is performing just fine for my needs so I don’t have much incentive to bother doing anything extra atm. Just figured I’d post it because I said I would. Jumbo frames really is the most important bit
Synthetic benchmarks are not indicative of real world performance, either.
Yeah that’s true. During actual copies, it has a habit of spiking up and down by about ~100MB/s for my setup. Here’s some pics of actual copies with a zip folder of ISOs:
Reads from NAS:
Writes to NAS:
Due to caching effects, fragmentation, and external factors the performance of the array and network are going to change over time.
Kinda love threads like this because they end up all over the place.
You can (and should), think of ZFS like a street legal racecar. It’s unlikely you’ll see full performance out of the box because like the car, there are way too many factors and quite frankly tuning is part of the fun.
Where a lot of threads end up is people with hardware and a not much more then a general idea of what performance they expect, are surprised when people who DO know, aren’t necessarily willing to provide it for free. Especially when it comes to Enterprise level hardware. At least give credit if I’m going to fill the RFP for you. That reddit thread is a perfect example of this. The problem is actually in the output presented but the OP has no idea what they are doing.
Like the racecar you can go insanely fast by ripping everything out - you’ll also die in a crash. Your data will suffer the same fate in a poorly planned filesystem.
It’s also unlikely you’ll gain much (any) performance by tuning unless you’ve taken the car apart and put it back together a hundred times. I do not recommend it for beginners.
After THREE. YEARS!!
Of trying to figure out what the HELL the problem was.
Someone asked: “Is DeDuplication on by chance…?”
I KNEW it.
After literally THREE! YEARS!
NO suggestion before even made me feel an iota of hope.
(In fact, many were downright stupid).
Sadly, that’s the 1st binary-switches even asked about 1 way or another.
…and SOLVED it!
I’m not saying it’s ‘as fast as I’d like’ (is anything ever…?)
But LOCAL transfers at 1.5MB/s per drive?
8 drives aggregate of 12MB/s ?
Anyway … I hope this helps someone else one day…
DISABLE DEDUPLICATION…on the dataset.
(Storage, Pools, Edit) …
I thought I usually say that when talking through zfs stuff? oh well
We should put together some ZFS troubleshooting boilerplate at some point.
This topic is resolved so I’m making a new thread
