So I have a bunch more questions on NASs and FreeNAS in particular..
So for my NAS the parts I will be using are:
Mobo: Asus H87i-PLUS
CPU: Intel i5 4670
Ram: Any 16GB kit
Case: Fractal Node 304
PSU: Corsair 430w 80+
And i'm not sure on the HDDs yet, but it does into one of the questions
Questions:
-Can I add HDDs later on in a plug and play style and not have to reformat the whole new volume?
-Can I add an SSD for caching later on?
-Can I use one SSD for up and down caching instead of having 2?
-What does zfs mean and what is the optimal raid setup?
-Is the RAID setup involving 5 drives that Wendell went over a setup where you have 2 drives for storage, 2 backup and one as an instant replacement if one dies? or is it something else?
-What RAID setup should I use?
-Wendel was explaining how if you have 8GB or less of RAM then the system can just hang, will having 16GB make sure that this never happens? The way he said made it sound as having more RAM just made the day where the hanging comes be way more in the future
Thanks for all the answers :)
When Wendel mentioned the system potentially hanging with only 8GB RAM he was talking about using the De-duplication feature. It is RAM intensive.
I'd say just get 16GB RAM from the start, that way you won't have to worry about system limitations as you add drives and do more with your data.
Western Digital Reds are supposed to be good for NAS systems. Definitely consider them when you're selecting your HDDs.
ZFS is the filesystem for FreeNAS. It's a bit different from what you use in Windows, Mac or Linux. It won't take took long for you to get your head around it.
Background info:
- Official documentation (worth referring to) http://doc.freenas.org
- Important topics on ZFS and things to be aware of http://nex7.blogspot.com/2013/03/readme1st.html
Tip: Make sure you have copies, elsewhere, of all of the data that you put on your FreeNAS system. Especially as you are still familiarising yourself with the system.
Hmm, well after reading the blog spot post and being informed and scared i've changed my mind on a few thins haha.
I think I might just buy all my HDDs and SSD at once just to limit any possible problems.
And speaking of HDDs and SSDs , how many should I get and should I get the SSD?
Also, if I got the recommended 5 number of drives, would only 2 of the be used as actual storage?
And do you think I should still get the SSD for caching? Or would it be a bit too difficult and not worth it to setup?
I've also heard that ECC RAM is a must but is that so? Coz that would mean that I would have to buy a special motherboard an everything which would be a huge pain..
If you are using free NAS in a production environment ECC is a must.
Bad ram will lead to bad files and bad backups.
No matter what ram you use you really need to run men test for 24 hours.
I haven't really used freenas, but I can probably answer a couple of your questions.
As far as I know you cannot add new disks to the array without having to format it, it's the same as RAID in that sense.
As far as how much space you'll get from the disks you use, basically if you have 5 disks you could have no redundancy, meaning that you will have all the space as usable but if a disk dies you lose everything. You could have 1 disk redundancy which means you will have 1 disk less usable space but you can lose a disk and not lose any data. And you can have 2 disk redundancy so you can lose 2 disks. With RAID-Z or whatever freenas uses (it's not really the same as RAID5 or 6) you can have any number of redundant disks and depending on how much capacity you plan on having you may need more than 2 disks for redundancy.
I can't remember off the top of my head but there's a formula for working out how many redundant disks you should have for a certain amount of storage. The problem is that as the capacity increases the chance that another disk will have an error while the array is repairing itself becomes very high.
Ahh ok that makes sense, I think i will probably go with 4TB WD Red drives.
But the other thing I was wondering about when it comes to this RAID topic is the fact the Wendel had 5 Drives, 2 as redundancy and 1 a "parody", what is the parody thing?
Hmm I might look into that then.. It won't be used in a mass production scenario, but I will probably put some home music production backups on there, but primarily it will be steam backups, tv shows/movies, photos and videos, music, programs (pc programs) and then a few odd things like music production backups (all in order of size on disk).
Would I have to get a special mobo and cpu to go with that ram though? Or would it work fine with my h87i and 4670?
He probably said parity but that still doesn't make any sense, so I'm not really sure what you're referring to. But it sounds like you're talking about a RAID 10 with a hot spare. There is more than one type of RAID configuration you can use so you really need to look at them and figure out which, if any, are what you want to use.
I don't use RAID or RAID-Z or anything like that because they are really built for availability, as in they're designed to keep working if there's a failure so that services reliant on the storage keep working. They're designed to handle a certain amount of failure after which you lose all the data, so they're designed to be used in conjunction with proper backups and are really only useful in enterprise environments. I see a lot of people wanting to set up RAID1 or RAID 10 which creates a mirror or each disk, but really they'd be better off just using a backup.
But like I said figure out exactly what you need from your storage. ZFS seems a lot better than using a regular RAID setup (although the performance isn't as good) as it has error correcting and stuff like that to prevent data corruption. in my opinion, unless you need your storage to always be available, then disk mirroring type RAID configs are not really worthwhile and using those extra disks for backup will be much more useful.
If you make an array with 5 disks and have one or 2 for redundancy bare in mind that while you won't lose any data if one or two disks fail, if you lose more disks than you have redundancy for you will lose all the data in the array. I would suggest if you do something like that to have at least two disks for redundancy depending on how much storage you end up with.
There are some other options aswell, if what your storing is mostly static, like a media library or something (static as in you're not writing to it frequently) then you might want to look as snapraid. Basically it lets you set up disk redundancy but you can add disks at any time and if you lose more disks than you have redundancy for you only lose the data that was on those disks. This is what I use and I've been pretty happy with it. But if you are writing to it frequently then it's not very useful.
I'd recommend going for an ECC based solution. Are you buying the components or are you using what you already have?
If you are buying them consider the ASRock Intel Avoton C2750 that was in Wendell's video. You should be able to get it for the price of the Asus motherboard + i5 that you referred to in your post. That way an ECC based solution won't really cost you more than what you were thinking of anyway.
I don't use ECC RAM on my current budget FreeNAS system, but I only use it to store copies of media files i have elsewhere. I just used whatever spare motherboard i already had (an ASRock Quad Celeron) just to get a feel for how it works. Once I start putting important data on it or start using the OwnCloud, email, and crash-plan plugins, i will replace the mother board/CPU and RAM with an ECC based solution. Hopefully by the end of the year.
I'm new to FreeNAS. I'm just using my current set up as a tester so I'm only running two HDDs mirrored.
As far as RAM I only have 4gb in my freenas box and havn't ran into any issues, and I run Plex media server on it, as well as Crashplan. So 16gb will be more than plenty.
I run ZFS. I ran it for 3 months as a test to make sure it was stable and never had issues. Now I have it set up and use it daily.
So after a lot of thought I have decided that reliability is going to be a must and this is goin o be my media server AND mass backup system so I need it to never fail under natural conditions.
Oh yeah and I was talking about parity, whatever it is haha.
I checked PCPartpicker NZ and there aren't any server boards in the country that are ITX size at least so i've decided that I will spend a good chunk of money on this build instead of the old 'reasonable amount' which included the h87i and 4670 from an old gaming rig.
So what parts exactly should I use?
I see that newegg ship to NZ now so if you can find any good mobos/CPUs/RAM on there then that'd be awesome, or direct to what to look for exactly.
I'm not too keen on the atom so i'd rather spend a bit more and get what I guess is next in line which would be a Xeon?
For the RAID/ZFS or whatever HDD setup, do you think 3 storage and 2 redundancy would be fine? Is the only reason to have more redundancy to make sure that you have enough time to run to the store for a replacement before the other one craps out?
I basically just want reliability and not needing to mess around much when it breaks-ness if you know what I mean?
I would rock a supermirco board.
I found this motherboard and thought that it looked pretty good. Has ECC support, 6x6Gbps sata ports, and it looks like it has a dual nic along with a single one (intel of course) and i was wondering if I could use then in tandem, like as a 2Gbps port?
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157466
Did you look at this board:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157475
Its actually designed for small to medium NAS builds and you do not have to add any extra sata cards, it includes the processor which uses passive cooling, the only thing you have to add is RAM.
ZIL/L2ARC cache drives are not going to benefit you unless you are using large database files which need constant access read/writes.
When I build these things I use Supermicro boards as they tend to be more robust then Intel boards,a QC Xeon, with 8gig ECC to start.
Since most do not think of a backup plan, I would recommend a raid 1+0, with a second set of drives as a swap setup.
Example: 4x6TB drives mirrored and striped, then a second set of 2x6TB for swapping depending on how much data you store, and add.......
I use an SSD for jails/plugin data alone and not store it on the actual raid array it self, that way if you ever have to rebuild the array, you do not have to deal existing plugin setup/configurations.
Also you can not add drives to an existing zdev pool, you can only create another separate pool aside from the initial one created, they are working on being able to add drives to the existing pool, but its still in the dev stage.
Hmm, that motherboard is quite expensive though..
I won't be using anymore than 8TB-12TB over the next 5-10+ years that I will be using this for most likely.
I definitely want to be able to use the NAS as a backup drive and also as a regular drive with single copies of things so for example:
Movies and TV Shows will be on the NAS only.
My music will be on my PC and NAS.
My Photos and Videos (from cameras) will most likely be stored on there only as well, which is the main concern for safety, but I shouldn't need it anywhere else because ZFS or RAID or whatever should be the backup because of the redundancy drives.
How often do drives break though?
Or should I REALLY have a drive on my PC as the original of the Photos and Videos?
The TV shows etc don't matter as much as they can be re attained, but the photos/videos obviously not..
Well,
Yeah the board is a little expensive, but you still have to buy the processor for the other one, now there is a cheaper one the 2550 which is a QC, and if its just storing and not trans-coding, and you can get that one for like $250, its the 8 core your paying for on the other one.
Building a second NAS for offsite backup is not always an option, installing a $1200 tap drive is not an option, and unless a person wants to invest time and money into a Blue-Ray-DL burner, at most that's 50gig a disk, not worth it.
Raid can be hardware or software, ZFS is software raid and does a real good job for what its designed for, any drive can fail at any time, even $400 SAS drives, with that being said, always have a backup plan
So just plan it out, and you'll be okay