I need Raid help!

Hardware HDD's & SSD's
1

In November/December this year I will be building are new system Ryzen 3000/2000 based system and turning my I7-2700k into a file server for my vast collection of music and other media. I have landed on purchasing 3 4tb seagate drives - Link - I am trying to do this low cost as I am quite humble and have spent the last 2 years saving for this.

I was wondering what would be the best way to do raid that is not hardware Raid I am willing to learn of course and I have spent the last few weeks looking into this doing my research on the topic and decided I would go with RAID 5 though I still am not totally set on the idea of it. I was also wondering would if possible to do like a RAID 5 + 1 as in have the 3 drives in raid 5 then have those drives backed up to a single 8tb drive - Drive - or would it be better to just do a RAID 6 with 4/5 drives at that price point I am trying to be safe/cost effective as much as I can I will have data on these drives that is very precious to me up till this point have been doing monthly manual back ups.

Please give me some good realistic advice advice on this I am very inexperienced when it comes to RAID solutions though I have worked with computers my whole life near enough I just never touched RAID.

Just some extra info the Mobo for the I7 is an asrock p67 pro 3 it only has 2 sata 3, and 4 sata 2’s I don’t mind running the 3x4TB drives on sata 2 as I want peace of mind over performance.

Also was looking into shucking a drive from - Drive -

2

You’re probably looking for something like RAID6 which is RAID5 with an extra parity disk.

1 Like
4

was thinking more towards raid 6 due the better redundancy do you have any more advice?

5

So if you have 3 drives and you want 2 drives of redundancy you just have two mirrors of the one drive. This is actually really cool because you don’t need anything special. Any one of the drives you can read in another computer with nothing special.

I love freenas for this type of use case. How much ram does your i7 have? You could run raidz1 (2 drives of storage and one drive of parity) and with the other drive for a backup, this is an acceptable level of redundancy.

zfs makes it easy to do snapshots so if someone deletes something you can roll back pretty easily. T his is fairly easy to setup on freenas.

you can of course do a 3 way mirror and have 1 drive’s worth of space but 2 levels of redundancy. This makes more sense than trying to do raid6 with only 3 drives. It’ll be faster and less overhead.

raidz2 is an option, but… don’t do it. It doesn’t make sense with only 3 drives.

the snapshots option is good for “safety” as well… if you get crypto’d or something, you can roll back to a prior zfs snapshot.

be sure to configure snapshots that makes sense… I’d recommend daily or twice daily, and to keep about 1-2 weeks of snapshots. generally if something bad has happened you notice sooner than a week. and if not you have your monthly backup still.

If you want to use the one large drive as a ZFS formatted volume, zfs send will make syncing the changes extremely insanely fast as well, which is a nice bonus.

Unraid is an option that is perhaps easier than freenas, and would work fine with a 2 way mirror (not raid6, you don’t want that really with just 3 drives. a 3 way mirror still means any two drives can fail). The snapshotting is clunkier on unraid imho. Unraid is also not free.

3 Likes
6

Currently 16gb.

I have budget for this $500 so I could have upto 5 x 4tb drives. I just gave 3 x 4tb as example, trying to lower the cost but I could try and get 5 though that would be hurting my budget for the Ryzen Build. To give you an Idea of what I am planning to get -

jpeg%20for%20lvl1tech
jpeg for lvl1tech.jpg1205Ă—657 103 KB
-

I had the idea of having 3 x 4tb in a RAID 5 then backing that up to a single 8tb if thats possible though looking at it might be smarter to just get more drives and do a RAID 6. What are things I have to worry about really RAID like how much does sudden powerloss really screw you over? What happens when the OS drive SSD fails that has the software raid configured is the RAID lost or is salvageable.

I will always be doing monthly manual backups for photos and other important files.

I was looking at Unraid but I want to use freeNas as I at the same time will be switching to Linux on the I7 been telling my self for years I need to switch this would be my way of really getting into Linux also.

7

Soft Raid will have to resync with surprise no power. Zfs would be fine. 3 drive raidz1 sounds like what you want. It’ll be more tolerant of power failures. It doesn’t need the same type of resync and does not suffer from the raid write hole

Zfs is very reliably portable between machines too – zpool import works great. Md softraid on Linux is pretty good too though.

2 Likes
8

Thank you for the info I will look into Software Raid on linux what is the best distro for this, I assume that its similar but better to Windows storage pool. I am also really looking into freenas as this really interests me. Also how easy is it to migrate an array lets say 3 years down the road when I change the hardrives or other hardware. How is updating the software for freenas?

9

been using zfs for about 10 years now, it’s been really solid. The only thing is not to over fill your volume, it can be dangerous when you have snapshots lurking.

you can very easily copy, rsync, or zfs send your data to another volume via the network or whatever.

windows soft raid and storage services doesn’t have anything on this. The very first version of windows home server was actually quite good but microsoft has cut a lot of the good bits, like shadow copy, out of modern windows.

if people really understood what they lost from win7 > win10 RE shadowcopy, there would have been riots in the street.

2 Likes
10

As per @wendell above ZFS for the win.

But… read up on it before you create your pool and be sure to ask questions if you aren’t sure.

ZFS will quite happily let you shoot yourself in the foot during pool creation by making dumb choices. That isn’t a ZFS failing but rather a lack of friendly or sensible admin tools to guide you away from dumb choices.

1 Like
11

FreeNAS has a gui so updating it is easy, But i would not try anything Beta, they had a beta or something once with many many user requested improvements, And in the end they went back on most of it. Thoe it was more like a new branch with its own name and not just a beta.

Migration of your data pool between FreeBSD (os of FreNAS) and Linux is possible if you stay away from distribution specific flags. However i have not tried it myself.

Replacing a drive is quite easy, and on freebsd you can even use hot spares. (meaning they are ready to replace a drive that falls out.
One of my favorite things with zfs drive replacement is that it only writes the HDD blocks that are used, instead of writing every block like MDADM does, This can drasticly reduce replacement speeds.

I just wanted to add something i feel should be mentioned when discussing raid1 vs raid2:

If you are worried about data degradation, raid5 or raidz1 means you have two copies of the file. so when one is degraded there is a 50% chance the filsystem will keep the right one.
raid6 or raidz2 would give you 3 points to check and the system would reliase that the one that is different from the other two is the corrupted file

But if something is mission critical you can use the copies function on zfs to keep more copies of files in a specific dataset to compensate.

Also since 4 dayes ago the expanding of a zfs vdev (raidz on Zfs On Linux) is in alpha, (In case you want to expand in the future).
They are calling it reflow, Wich i think is becose it is not a proper block point rewrite. So you wont get the same size utilisation if you decide to expand later.
But my source on this is abit uncertain, So feel free to correct me (Anyone).

1 Like
12

I don’t have any correction, but I always saw the reflow as a way to balance which data was on which drive.
When one Adds a vdev to a pool, it will be basically formatted, then new data shared between new and existing vdevs (according to varied rules) but the new vdev starts empty, then fills over time.

With the reflow, I got the impression that all the existing data is checked, and some of it written to the new vdev straight away, rather than waiting for new data to be biased to the new device?

1 Like
13

i’m glad for the input, but i belive you are talking about rebalancing a pool (between multiple vdev), According to the FreeNAS forum it is a time honored tradition to move your files around to cause a rebalance, or just make a new dataset on the same pool and move the files.

Reflow in this case is the upcoming ability to increase the size of an exsisting vdev not adding one. So i belive you are talking about something different. This is 4 dayes into the alpha of ZOL

Part of my intention mentioning the term reflow was that i struggled finding any info about the upcoming ability to expand a vdedv until i noticed the term being used.

1 Like
14

Okay, I must have misconstrued what Matt Athens was saying back when he was starting the raids expansion a couple of years ago.
I prefer mirrors myself, so didn’t pay too much attention

1 Like
15

there is also a chance that i missed something about the new stuff, after my disapointment i skimmed it.
I should probably read it properly at some point so i don’t unintenionally spread misinformation.

1 Like
16

Thanks for the input guys I do have a lot of choices to make, like I stated I won’t being do this till November but I like to be prepared and have time to learn about things I already have freenas on Windows VM.

My actual use case for this will be a little more than just file redundancy, I really like the ability to run VM’s and the plugins so far looks amazing :).

Couple more questions to things I am struggling to find answers for:

Snapshots how much space do they take up?

SSD cache is it really needed or should I use that as fast storage for VM’s?

Can I raid 1/mirror my Freenas OS boot drive?

How do I backup Freenas Configs and should I just do this to USB’s?

3x4tb in raidz1 or 6x2tb in raidz2(I will have get sata controller suggestions are welcome)?

My I7 Freenas will have no networking capabilities (no wifi/router is too far to be wired) so I will have to connect it through Ethernet to my second PC as pass through which has a wifi card, will this be alright setting up?

Currently have 16gb of ram for the I7 2x4gb and 1x8gb should I invest and get 2x8gb upgrade to 24gb?

17

I’m not an expert on FreeNas, but if you use it with ZFS I can give a few pointers:

1- space for snapshot- starts off as practiacally 0, but grows as it references data that has been deleted. It’s pretty much instant too, which some other snapshotting isn’t

2- ssd cache - if you have a small ssd you can give it, you can speed up the read (L2arc) or the write, or both with 2 SSD’s but it’ll only cache up a few seconds of writes, so depending on how you use it, might not really help with sustained writes, more help with sporadic writes

3- mirror boot USB - yep, if you have 3 USB sticks, you can use one to install FreeNas onto the other two, if you want, but you are free to install on a single USB/HDD/SSD. FreeNas runs from RAM once booted, to doesn’t matter what it’s actually installed on (as long as it boots)

4- backup configs - FreeNas copies it’s config to the first dataset it creates, so you could do a fresh install later, and find your old config comes back, whether you want that or not. I’ve heard it copies the data to every dataset it creates, but havn’t tested

5- narrowz1 vs widerz2 - your choice because it’s your money, and it depends on space for expansion later, but generally 6x2tb will give better read performance, better durability, possibly faster write speed, or worse, depending on the cpu (z2 calculations are slower than z1, but more drives means it’s waiting less for the drives to be available for writes) 6x2 drives will allow a little longer to replace one if one becomes faulty too, but you should replace failed drives as soon as practicable regardless, rather than put it off for later.
3x4 drives might be cheaper to buy, and might use less electricity

6 - network via shared other computer - I have never tried this, but I can only guess that it would harm performance… it should work, but you might have a harder time setting up a connection, depending on how the internet connection sharing works on the middle machine, and how well it’s DHCP works.
I have definitely used a FreeNas box without any internet/router connection, directly connected to a computer, and it worked very well as an offile device

7 - 16gb ram okay? - yeah, that is plenty. FreeNas / ZFS have built in RAM caching, which you can adjust in the FreeNas settings to reduce the amount, but the more RAM it can use, the better cahcing it can do. I’m sure FreeNas can run on only 8GB, regardless of the data size, but it can impact performance if you have large or many data drives with too little RAM.
I’d say go ahead with the 16gb, and every now and then check the performance as you go.
It would use as much ram as you give it :slight_smile:

2 Likes
18

Would it possible to back up the storage pool to just 1 drive, example being 6x2tb in raidz2 = around 8tb of storage could I then back up that 8tb storage to a single physical 8/10tb drive.

19

Yeah, you can do that. It might take a while, but can definitely do that

1 Like
21
My actual use case for this will be a little more than just file redundancy, I really like the ability to run VM's and the plugins so far looks amazing :).

Virtual Machines under FreeNAS is in my opinion lacking, VFIO passtrough for instance is not a thing. But dont underestimate the abilities of a freeBSD jail, They are great.

Snapshots how much space do they take up?

The size of a snapshot is the difference from its source, lets say you delete 1mb, you wont get that space back becose you have a snapshot of the filesystem with that 1mb.
Additional snapshots wont take up more space, unless they are different from both the source and the other snapshots.

SSD cache is it really needed or should I use that as fast storage for VM's?

Unless you have an absurd amount of ram, in most cases a ssd cache will just slow you down, However the ZFS intent log (often called slog or separate log) can dramasticly increase synchronous write speeds wich is what you should be doing for Virtual Machines or sql databases.
To be very clear, this is a write log not a read cache (l2arch)

Or if you love pictures like i do :smiley:

But dont run a slog in your RAM, unless it is a physical ramdisk with a battery backup.

Thoe it might just be easier to run things like virtual Virtual Machines on the SSD and backup it up to the raid trough automatic snaphots

Can I raid 1/mirror my Freenas OS boot drive?
Running two usb drives in mirror is actually the recommended way according to the FreeNAS forum. 4GB is enough, they are dirt cheap.
3x4tb in raidz1 or 6x2tb in raidz2(I will have get sata controller suggestions are welcome)?

raidz1 vs raidz2 i answered above, but i can also add that more drives will increase your speeds. Your power usage, noise and heat production.
For the recommendation of sata controller i can point you to the FreeNAS thread about it, but keep in mind they tend to recommend used hardware raid controllers and then flash the firmware on them to disable the raid stuff:

Would it possible to back up the storage pool to just 1 drive, example being 6x2tb in raidz2 = around 8tb of storage could I then back up that 8tb storage to a single physical 8/10tb drive.

Yes, ZFS even has an export function that only sends the changes since the last send,
So you wont haveto compare every single file on the drive like with rsync. Meaning that if the changes are minimal you are done in two seconds with the backup to your single 8TB drive.
But send and recieve is a ZFS function so you would haveto run ZFS on the 8TB drive aswell.

I skipped some questions becose either i had nothing to add to the answers by @Trooper_ish or i just don’t feel qualified to answer them.

2 Likes
22

Brilliant reply appreciate the effort and sharing of knowledge guys.

I am not gonna go with the 6x2tb as it will be annoying in few years time upgrading it, I am going to stick with the 3x4tb as this gives me space to grow and then upgrade in the future depending on my needs I imagine though I will push to a 6x4tb over time and either raidz2 it or the equivalent raid60 but I will cross that bridge when it comes to it.

I don’t want to boot from USB’s I have had so many just fail on me over the years its not worth it have 1 120gb SSD spare and I will just pick another 1 up and mirror them.

I have plenty of 16gb-32gb USB’s lying about so I will use these to back up configs.

I will also pick a cheap 480gb SSD for my VM’s, If your curious to what I will do with them I will most likely run game servers from time to time, Cloud Pc when I am out and about (Wifi needed of course)

I think I will upgrade the ram from 16gb to 24gb the more the merrier also ram is cheaper than ever right now.

I have had the I7 2700k running at 4.5ghz at 1.37v for about 8 years now should I downclock it to extend longevity(4ghz at 1.28v) of it or just leave it?