I noticed my ARC hit ratio is pretty low. Anyone know why?
My system:
ASRock C2750D4I
16GB Kingston 1600MHz DDR3L ECC
4 x 4TB WD Red + 2 x 4TB Ironwolf
3 mirrored vdev’s. 12TB raw storage.
On FreeNAS-9.3-STABLE-201501212031
No periodic snapshots.
The uptime is 21 days
And one of the pools is 84% full.
One of the pools have recently been physically removed but not destroyed, but in the picture you can see the ARC hit ratio steadily declining.
At least from what I've read in the doc and on the freenas forums having a pool over 80% capacity will steadily decrease performance pretty much just like you are experiencing. I would put that other pool back in and then see how it performs if you are experiencing problems.
I recently cleared 4tb (the pool was 82 % full) off of my media server and POOF performance returned in a big way.
Yup FreeNAS should be warning you that a pool is getting full. I think it starts nagging over 80%. Do you have email set up? I have FreeNAS sending email to my gmail account using the gmail server. I have an application password setup for FreeNAS, so it doesn't use my google password.
Time to clear out stuff you don't watch. If FreeNAS is taking regular snapshots, try weeding them out, reduce the retention duration.
I have had to clear out a bunch of snapshots before. Seemed like a good idea at the time, but i realistically never rolled anything back, I would just fix it.
I don't think it's because the pool is at 84%. That happened last month. But if you see the graph the ARC hit ratio is low in aug/sep/oct. I Asked Wendell on a PGP stream and he said it needs more RAM? I destroyed the pool I removed, will see tomorrow if it has an affect.
I think you have enough RAM. I ran FreeNAS for ages with 8GB of ram, and a 9TB pool. I was constantly using SWAP and many applications in jails ran quite slow. After upgrading to 16GB of RAM, my graphs look more like yours. Only a small amount of swap gets used every now and then, because I run quite a few jails. It looks like you doubled your ram back in feb. Good move!
Thinking more about it, the ARC is the memory read cache. If you keep watching different media files, this could easily explain the ARC percentage dropping. I wouldn't worry about this. It just means you are reading files from disk and not from memory, which will be a bit slower. But the purpose of a NAS is to store files on HDDs. Memory caching is a bonus. You can only get 100% if you have enough ram to contain all of your pool data! Which is unrealistic.
A lot of ARC hits will also be metadata, but if you're not accessing the same data often you won't see any ARC hits, the more stuff that is read the more stuff is pushed out of cache, so when you re-read it it may not be there anymore. Basically, don't worry about it, for a media server cache is pretty useless.