Hi @mountain_cabbage
The TS and my own config has such a large amount of RAM due to ZFS requiring a 1 GB per 1 TB of storage management system, for things like hashing the data for bit rot comparisons, duplication checks and the like, as it's a software RAID rather than hardware RAID. I'm pretty new to it all, but https://en.wikipedia.org/wiki/ZFS covers it very well.
The problem with using large drives is the time it takes to replace them if/when there's an error, so the best practice is generally to have a larger array of smaller drives, to compensate for such contingencies. Of course, against such judgement I've gone for 6TB drives!
RAID 5 is such a case I think, of the rebuild time potentially leading to further disks failing, as say with 10 TB disks with 5 in RAID 5, one breaks and the remaining four are all being read to rebuild the striping for the replacement, leading to potentially another in the chain breaking. That type of RAID doesn't have much fault-tolerance for losing more than one disk. It's one of the reasons why I liked ZFS so much, even though the OS and file system are quite foreign to me. RAIDZ2 allows for 2 disks to break before losing data; so when one fails and it's being rebuilt, if another fails I've got a greater chance of recovering without another failing.
RAID 6/RAIDZ2 in your case for larger drives, may be a better way to go as it's double distributed parity, allowing up to two failed drives. RAIDZ3 is relatively new and was implemented for larger HDDs, like 6-10TB upwards and allows 3 drives to fail, for the long times it takes to replace them in the RAID.
What you could do is build two NAS systems and have them sync the data between each other, so you have a failover state for your storage and some tolerance for one system failing but business operations continuing with the other system whilst you rebuild/repair the faulty one. You can have as many as you want/have the resources to implement and support.
Cache drives also promote data integrity as well as write speeds to the NAS, as it effectively caches the user data and then writes it within the storage. Depending upon how much data you're expecting to be written to the NAS and the overhead of losing native drive ports/PCIE slots for NVME or M.2 drives, it a use-specific case I think.
Please excuse the all-over-the-place replies to your questions and thoughts.
Are the 10 TB and 2x3TB HDDs to be configured together in the RAID or are the 2x3TBs for O/S and/or other systems RAID?
Say all HDDs as storage, performance and no fault-tolerance due to sync'd backup and 30 day limit - yes RAID 0 would do fine. Although if the data is critical within that 30 day limit, I'd run pre-production tests and have a pre-production pilot month, to highlight any issues. Once that's done, I'd still run regular tests to ensure that the whole system worked throughout those 30 days or build in some data recovery for the potential breakdown of both systems.
RAID 5 for 3x10TB and 2x3TB HDDs would be better overall from the outset, imo, but certainly it works within your desired future system.
Remember RAID cards can be a point of failure too and they don't necessarily deal well with bit rot, although the lifetime of your data storage needs is quite short that this shouldn't be too much of an issue. I'd be more concerned with the quality of the HDDs and the likely mileage they'll be racking up over the months due to the filling and purging of data.
Multi-core CPUs are best regardless, ECC RAM is a recommendation although more essential for certain NAS OSes - which is where the specs diverge somewhat based upon needs.
Non-ECC RAM and Buffered/Registered ECC RAM is more prevalent and cheaper, so it depends upon the Motherboard specs as to limits and types required. For most I think 8 GB RAM is sufficient, but ZFS-based require 1GB per 1TB of storage ratio of RAM. More RAM generally helps with whatever you're doing, but 8 GB should be sufficient for your needs. The reason I've gone for full-on RAM in my build is due to the lack of choice for RAM for my choice of M/B; luckily I found the Crucial sticks rather than the only other option, saving myself spending more. Ideally it's better to ramp up the specs as requirements change, but I don't mind the initial outlay knowing that I'm covered from here on in RAM-wise!
I did research several types of CPU, M/Bs, etc., but I have to admit I liked the native 12-SATA drives of the ASRock C2750 Intel Avaton Mini-ITX Motherboard than most other Xeon-based Motherboards.
- ASRock C2750 Intel Avaton Motherboard
- 64GB ECC RAM - (4x) Crucial 16GB DDR3-1600 ECC UDIMM VLP
- Toshiba X300 6TB 128MB Cache HDD - (4x)
- CiT 2206 Case
- Cooler Master V650 PSU
Oh and for the 24-Port Gigabit switch - if you're likely to have that many users feeding data to your NAS, you may want to have a managed switch to control that throughput. Maybe someone with better networking chops than I can offer better advice regarding that. I've got an unmanaged Netgear 8-Port Gigabit switch at home, that suits my setup fine.
