I am currently in the process of putting together my new NAS-Server. Since I recently upgraded the CPU in my main rig, I have a Ryzen 5 2600 CPU that I could use for the build. I have an option to get a used ASRock Rack X470D4U2-2T Motherboard for a decent price. I plan to upgrade my main PC next year and I also want to have faster than 1G NIC to that PC (shouldn’t be a problem with all of the new platforms having at least 2.5G). The Motherboard has 2x10G Intel X550 NICs. Would that CPU be fast enough for 12x 3.5" Seagate EXOS/Ironwolf HDDs (connected to two Dell PERC H310s / IT-mode flashed), 8 of them as backup drives and 4 of them to edit photos and videos off of, and saturate at least one of the 10G NICs at the same time?
My second question would be how much RAM do I need? As OS I will be using TrueNAS with 3 RAID-Z1 pools. One of them will have 4x4TB and the other two will be 4x10-14TB each and one of those latter ones will function as a backup to the one I will be editing off of. Would 64GB be enough or would I need 128GB? I also have the option to add NVMe SSDs as cache if needed.
I wouldn’t worry about the CPU. And when talking photos and videos, you probably won’t use file system compression anyway, which is the main CPU hog for ZFS. My NAS runs with a 5900x but I don’t see any troublesome CPU load when moving stuff at 1GiB/s. I doubt a Zen1 2600 core that can boost to 3.9GHz will have any trouble.
For normal operation, 4GB-8GB is plenty for truenas and ZFS. But you want to cache metadata from your frequently used files along with actual files cached that are in frequent/recent usage. With photos and especially videos, you don’t benefit from filesystem compression and thus compressed ARC. Memory is probably too expensive to cover most media, so I recommend going for 64GB and an NVMe as L2ARC (akacache). Get 2x32GB sticks, so you can upgrade later. And 1-2GB NVMe consumer drive (no need to go expensive here. I’m using PCIe 3.0 1TB for my pool)
I have no experience in live editing large stuff via ethernet, but latency and especially your editing software may not result in flawless experience independent from your network bandwidth. Nothing better than working off local memory.
You can make a pool with a vdev of 4x4TB and another vdev with 4x10-14TB, all in one pool. Easier management and probably more performance because ZFS uses striping across multiple vdevs. The remaining drives for backup. They are 12TB less capacity, but you never fill a ZFS pool to 100% (long story, expand when at 75-80%), but you can with your backup pool.
How large would the L2ARC have to be? I have not found any general guideline, only that it depends on the specific use case. Would 1TB be enough or already too large?
You have enough memory to not have to care about fitting metadata within the ARC, but having large frequently/recently used files being read from NVMe instead of HDD is very handy. 1TB is plenty. More cache can store more files. But with photo/video data, it’s all sequential reads so HDDs will still pump out 500+MB/s. It’s not like you are writing random 4k blocks all the time.
If you feel the writes are too slow or taxing on the HDDs, you can always add a SLOG or tuning some parameters afterwards. Same applies to L2ARC, can be added later without problems.