Considering SSD-based homebuilt NAS with ZFS, comments wanted

Hardware Hub Storage
1

I’m considering creating myself a little network attached storage, and I’d like to bounce my ideas here before starting the implementation.

This is for personal, in-home use only. The goal is two-fold, with the primary one being to have a more robust storage for my “long term file collection” of photos such, and the second goal is just to learn and have fun. I currently have these files on an SSD in my windows machine, but as I’m now moving to more of a multi-computer setup with also Linux workstations involved I want to move on.

The basic thoughts so far is to create a small, low-power machine runnig FreeBSD or possibly TueNAS Core with an ZFS file system that I share to my other machines. I

Computing hardware would ideally be something robust and low power, and I’ve been looking at ITX-sized Atom hardware with ECC support that is easily available in my country (Sweden). 'm trying to create something that is virtually silent, so if fans are needed, I want to keep them running slowly.

For the actual storage I’m now considering having a couple of SSDs running as a mirrored vdev, mainly to reduce the amount of moving parts, and to get the redudancy of mirrors. In total I’m looking at having something along the lines of 2 TB in this share, I’m currently using 1 TB and it’s growing rather slowly - so 2 TB should be fine for quite some time, even if using snapshots.

So… any generic comments or advice on the way? Any sources I’ve missed that have alreay created what I’m looking for? Bad idea to use SSDs in this way? Are the low power Atom boards powerful enough for my use?

Is it a good idea to use comporession for my use case? I guess that the MPEGs and JPEGs won’t compress too much, but I have other files too.

(And no ,this is not a replacement for backups, I still will want to take real backsups and move off-site).

2

This sounds like it would work just fine. For Atom CPUs, you probably want to use something in the C3000 line. With a mirror VDEV, even a 2 core Atom CPU would probably be enough, but check on pricing and how much you have in your budget. Nothing wrong with a 4 or even 8 core option if you want to do more with it.

Usually it’s preferred to boot the machine off of a different drive what you are giving to ZFS to manage. If you have a spare small drive that would work well, or you could even potentially use a USB flash drive (many of the ITX Atom boards have a USB port on the board itself for this purpose).

Yes, definitely. lz4 compression is very cheap on CPU, and it sounds like you aren’t going to be hitting the array very hard anyway, so this is worth it for some free savings.

1 Like
3

Thank you. Yes, I was considering booting of a separate drive, either M.2 or some simple SATA stuff. And yes, I was looking at the C3000-line as that is what my local preferred reseller of such stuff has in stock.

So far the only thing other than pure file serving that I’ve looked at is a media server to serve FLAC files to my Hifi setup… but I assume most systems would be able to handle that.

4

That should be fine unless it is doing transcoding on the fly. In that case, you might need a more powerful CPU.

1 Like
5

I’d take a look at proxmox instead of truenas or sticking with bsd. My experience with freenas was bad enough that I swore off its use entirely. I would also think about future use cases you may be interested in and expansion of the array.

2 Likes
6

Whenever anyone asks for advice about NAS’s I generally have them think about the future while building in the present. It’s certainly important to think about expansion when building your NAS to begin with.

In your case, I’ll echo that ATOM hardware is more than sufficient for your use case. As I’m sure that you’re aware, ZFS based NAS’s like a lot of Memory. So in this case, consider memory density. Instead of getting 2 small sticks of ECC RAM, I would get a larger single EEC stick (something like 8GB as opposed to 2 or 4GB) and run it in single channel for now. If in the future, you wanted to upgrade and expand your storage, you’ll want the spots available on your motherboard to do so instead of filling them up with relatively small sticks and having to scrap those as you need to expand storage.

Chassis can also be a concern, I know you’re going for low power and quiet, but might you also consider future expansion and adding your 2nd mirror, 3rd mirror, 4th mirror. plenty of options since you’re going for 2.5 SSDs. But don’t limit youreself to something so small that it becomes a challenge to add. Also if you’re staying low powered atom and SSDs you don’t need a gigantic PSU either, but keep in mind that native PSU connectors for every drive are preferred over pigtail extenders or multipliers. Those become another point of failure on a 24/7 machine.

Finally, a word on TrueNAS. If you go that direction then great, I happen to love TrueNAS/FreeNAS. But the topology of initial drive layout is something to consider. You seem to already know about how ZFS works so I won’t go into to much detail. You also sound like you know that Mirrored vdevs are the way you want to go. But just a reminder that unless you want to start from scratch and remake you entire pools, that drive layout will be how you add storage to your pool in the future. Furthermore, while mirrored vdevs are fine and they essentially provide per drive redundancy and decent performance, you are losing out on either the performance end or total storage capacity and also putting yourself in a position to lose everything in a pool if both drives in a single mirrored vdev fail. Of course this argument expands better when discussing larger layouts but I’ll try to get what I’m saying across.

Scenario:
Hypothetically, if you had a pool consisting of 2 Mirrored Vdevs that 4 drives total and half redundancy and half storage. Lets call the drives 1&2, 3&4.

Now lets say that instead of Mirrored vdevs, you had 4 drives setup in a RaidZ2. So that’s the same, half storage, half redundancy. They’re functionally equivalent. Let’s call these drives 1&2&3&4.

Now if drive 1 fails on your mirrored vdev setup, you’re fine. You can resilver your array by replacing drive 1. But if drive 2 fails during that time. Your pool is gone because 1 vdev is completely missing.

Conversely, in the RaidZ2 setup if drive 1 fails, you also resilver the array and if drive 2 fails during that time, you are still fine and can recover by replacing drive 2 and doing another resilver. In this scenario, your methods of a recoverable failure are much greater.

1 Like
7

I’m sorry but English is not my first language. Should I read that as a recommendation to use Proxmox even over “regular” FreeBSD or just over TrueNAS?

8

Just over truenas. Bsd or proxmox would be my picks

1 Like
9

You should benchmark this. With an Atom CPU and SSDs I worry that you might bottleneck on CPU in the (de)compression process.

It was older hardware but when I had a NAS built on AMD Bobcat E-350 my SSH, SCP and therefore the rsync transfers would bottleneck on the AES encryption and would limit the speed to about 30 MBps instead of 110.

So I don’t know what today’s Atom CPUs are capable of but definitely benchmark the compression to see if it works for you.

Also, I’ve heard 2.5 Gbps networking is becoming more common, and with SSD storage you can definitely use it. I’d look around to see if any of the boards have it (or 5 or 10 Gbps) if the price isn’t crazy.

1 Like
10

Released in 2011 and I couldn’t find support for AES-NI instructions set so that may be why any cryptographic functions involving AES were very slow on your hardware back then. With modern chips that’s not a concern. And Intel Atom C3000 series definitely has those extensions.

11

I just brought up the Bobcat to illustrate a CPU bottleneck. I am sure modern Atom CPUs are much faster, but then the SSDs are faster today too.

1 Like
12

Thanks.

I do have 2.5 Gbps on my main PC motherboard, but I’ve not yet found a lowpower Atom board with it. I’ll probably stick with regular gigabit for the time being.

13

?
i’ve had great experiences in truenas core. even had a dev livepatch a bug on my system relating to AD integration

14

I just posted some instructions on how to use Debian and btrfs for a NAS into wikis and how-tos.

It’s not complete, but the guide will get you a working reliable multi-disk setup with encrypted disks that you can SSH into and unlock remotely.

I haven’t had time to finish the guide fully - e.g. basic stuff on adding disks/removing disks changing passwords and so on is there.

Installing and setting up Samba and rclone, and setting up cronjobs for scrubbing data and so on is not there - that’s needed, but should probably be a separate guide - I’d imagine it’d be similar across btrfs/zfs/…

1 Like
15

I had bugs too and I wasn’t really doing anything but serving files. Ditched it and never looked back. Proxmox has been stable where freenas wasn’t.

16

I have three TrueNAS Core systems, thy been great over the years. smooth upgrades, the times i had problems its been a problems with hardware.

Make sure you follow the recommend/ supported hardware, and use a small SDD or SATA DOM for the boot drive. NO USB boot drives.

2 Likes
17

be careful with 2.5G on Truenas/ BSD my understanding the driver support is not there yet. That’s the big thing to watch out for on Truenas or BSD in general is driver support. Networking from Realtek is not well supported, it will work but you end up having horrible file transfer rates. On my BSD systems i dumped the on board networking for Intel and Mellanox for improved network speeds

1 Like
18

i have 2x1gbit lagg to the switch

19

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.