I’m extremely new to using ZFS and there are a few thing which I want to do. I will state the reasons down below.
-
Turn off ARC and all kinds of write cache.
-
Get a prompt before ZFS repairs a rotten bit of a file. Or if that is not possible, get a log of which file was “repaired”.
-
Optimize it for an all-SSD pool.
Now that I’ve stated what I want to do, I’ll talk about why I’ve made such “bad” choices. I live in India - where computer prices are ever so sky rocketing and where old quad core servers with mere 8GB ECC RAM cost at-least $400 (yeah it’s cheaper in USA but it ends up being the same cost-wise if I ship the said server in India, so not really worth it). The only reason why I chose ZFS is because of it’s checksumming and repairing features. All I need from a file system is for it to checksum the data being written in real time - which ZFS does, and for the file system to periodically check the whole disk for any kind of bit rot - which ZFS does.
Now, because of high prices of new & old hardware, ECC memory is out of question over here in India. I will be ordering a workstation for myself with Ryzen 9 3900X (yes I know Ryzen has un-official support for ECC memory, but I either want all in or nothing at all - no hanging in between for clear and blunt “I have ECC memory” or “I don’t have ECC memory” in terms of troubleshooting & support) for Deep Learning, I have it equipped with 5x 1TB SSDs with the intention of getting 3.5-4TB of available space in raidz1.
Reasons why I’m going with SSDs instead of HDDs
-
The storage pool will only be written once and is purely for archival purposes, so exponentially faster scrubs because of SSD’s read/write speeds.
-
Even if a disk fails, that rebuild speed even of a SATA SSD is a godsend.
-
Energy efficient
As I’m only using SSDs, the write speed impact when turning off ARC won’t be a problem for me. Also as shocking this might sound to the majority of you, the SSDs I’ve chosen are consumer TLC SSDs and I do not intend to use them as a fast editing NAS so there’s that. I’ll only be filling at-most 1.7 TB of data in the 3.5-4TB pool once it is all set up and writes will be at most 10 GB of files, only once a month. I can’t exaggerate this enough. I selected SSDs because I need to spend less time cringing + being nervous when a scrub occurs and in future when I need to replace a disk and resliver.
Talking about my intended setup, please bear with the insanity of my “intended” setup; I will be setting up a Ubuntu server VM and use zfsutils-linux
for the ZFS part of my setup. I will perform the optimizations which I asked for help in the beginning of the post and then setup the 5x passed through SSDs in raidz1. Then I will transfer all the important data I have to the SSD pool. I also have an external 4TB USB HDD and a Raspberry Pi 4B 4GB. I will set that RPi with Ubuntu server as well and use the single HDD in ZFS as well (this time I don’t need any of the above requested optimzations). That Raspberry Pi setup is just for redundancy as you’ll read about it more down below.
Reasons as to why I have opted for such bizarre setup.
-
Can’t justify ECC memory with my budget so I want to turn off any writes in RAM if possible. As, if I have understood ZFS correctly, ZFS will first write to ARC and then to the disks. So a bit flip in non-ECC memory won’t be detectable at the time writes. Also because I have pretty fast pool write speed considering all of it is just SSDs.
-
I requested to get a prompt before ZFS repairs/attempts to repair a file is because I have non-ECC memory. I want to make sure if the file is actually corrupt myself. If it is corrupt, I will let ZFS go ahead and if it isn’t, it’s most probably a RAM issue and I will try to see if a reboot helps. Or else, I do have a remote copy of the same file on the Raspberry Pi
-
If getting a prompt before ZFS repairs a file is not possible, I want to get a log entry regarding which file was “repaired”. I will perform the same verification as I mentioned in the above point myself as I have no problem doing a bit more work in the absence of ECC memory.
This will be my first ZFS build - as it clearly appears so. So any help/recommendations regarding my setup will be extremely well received. :’)