Nas Operating system

Networking Networking Software
, ,
21

I’ll let Jim Salter be my counter to the requirement for ECC:

http://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/

TL;DR: ZFS checksums data, so the only way you’d run into issues with data corruption is if you’ve got ram that’s so evil that it flips a single bit in every single byte that passes through it. If a single error occurs, the checksum won’t pass. If the checksum doesn’t pass, ZFS will rebuild it from parity/mirror.

That said, if you want ECC, don’t be afraid to use it!

22

To be honest HDDs have checksum too. And the main issue that ZFS is not going to fix is an application, say SMB server, that reads streams of data into memory and then calls operating system to write its blocks of memory to the volume X. No checksum mechanism introduced afterwards will fix the issue if the initial block of memory caused corruption. It seems like elaborated scenario, and ZFS actually can help to faster detect the issue, but still it might be days before you will actually decide to do full memory test because some random programs became unstable (as it all depends on what program will get allocated that bad memory address).

23

You’re vastly exaggerating the danger of bit rot.

24

One bit for a media file, yes. Every few bits for every 1 MB of a media file, yes.
One bit for an archive file, maybe. Every few bits for every 1MB of an archive file, no.
And yes, I’m over sensitive on that subject :smiley:

25

Antecdotal of course, but I have ran ZFS with no ECC. The world will not stop turning. No issues in over two years. Ive also ran it with ECC, no problems there either.

That being said, it all depends on how mission critical the data is. If you cannot afford even the chance of losing data, bite the bullet and go with ECC. For most home user NAS applications, non ECC will be fine.

1 Like
26

Every few bits for every 1 MB of a media file,

Every few bits for every 1MB of an archive file,

RAM doesn’t fail like that.

The main thing to understand is that ECC won’t protect you from bad RAM. You have the same rate of memory flaws (which is clearly what you’ve experienced) in ECC ram as in non-ECC. The problem is that you’re (understandably) mistaken in thinking that having ECC will protect your data 100% of the time from memory errors.

The only way to be relatively confident you won’t lose data is to follow the 3-2-1 rule with backups.

3 copies of the data
2 different types of media
1 offsite
1 Like
27

If an errant application corrupts a file via an SMB write or any other file sharing protocol, your recourse is to recover from snapshot or backup. It’s not meant to police applications. If it’s told to overwrite a file with a corrupt one, it will do it without protest.

1 Like
28

ok thanks guys
i have decided and its final :smiley:
I will chose one to learn at the same time it will be the way am creating this NAS

XD but what to learn now is the main question hhhhhhhhhhhh
linux ubuntu ?
freenas?
other options ?

xeon and ecc + motherboard are not expensive here , in dubai i can find very easily used workstations with older gen xeons ddr3 ecc ram etc…

29

Server memory, not just ECC

In general there is also an aspect that I have a larger expectation on those green PCBs that weren’t “painted” into colors, do not have RGB, and are not overclocked together with the CPU memory controller on their “normal” settings. And the same goes for the motherboards.

No, If you are lucky with a search you will find my description of my approach to backups. I’m only oversensitive in regards to ECC :wink:

Only for a sake of academic discussion…

To be honest if you are unlucky, it will “fail” this way. Not RAM, your files.

I simply take into account how usual server software is written (e.g. http server, smb server). Especially while steaming binary content (aka files). Between various layers of OS and application, in general, pieces of data is copied between buffers (buffer as how you would call an array/table of bytes used to store piece of file). Even if data is not modified, it might be copied several times between buffers or simply blocks. And almost in all layers, very likely, a optimization will be used to use a set of the same buffers over and over again (for the same purpose). @oO.o (That is what I mean). So that one bit error can start occur quite semi regularly.

So, if in general program code use the same memory (like those buffers), then why programs simply completely randomly not try to divide by zero :slight_smile: or any other stupid mistake that will signal the issue ?
The ratio of memory usage for program code to data is 1:100 at nearest. On servers, more like 1:1000.
So even if an HTTP server will crash on division by zero (just an example) first that you will think is “that must be that last update from week ago”. And before you will discover actual cause of instabilities you might lose some of your own time.

But like I said, academic discussion.
Also, because I assume we are all kind of talking about home scenario where half of those files will be copied from the machine with no ECC memory :smiley:

EDIT:
The one thing I failed to mention from the beginning: I do take into account an scenario that you take 3-5 year old parts and build a NAS from them and expect to use it for the next 3-5 years (sometimes if not always 24/7). So I seriously consider aging of such components like RAM. While RAM might not be the most stressed component we still need to member that most consumer memory modules are aiming for higher and higher clocks not necessary for longevity.

30

OP should probably build a minimal cost Xeon system and have a play and learn in the process. That’s what I did with an older Core2Quad system with an Intel server board (8GB DDR2 ECC) in an Antec mid-tower. More than sufficient to run XPEnology as a network storage device. If you want to do the streaming/transcoding and keep power down then you’re back to the modern SoC based systems.

The current Synolgy NAS devices tend to be less upgradable than the equivalent QNAP models though they are only the two brands more popular in the SoHo arena. For example I have 3 Reolink IP cameras setup and QNAP QTS supports them by model number so hardware compatibility was another factor in my choice.

As for bit rot my NAS by default runs a file check each month and of course it’s has RAID for redundancy which I’ve tested by ejecting the HDDs and letting the OS rebuild when I swapped from 3TB to 4TB drives.

31

I always thought ZFS is more or less exclusive to BSD and i didn’t ever use BSD (yet) and i didn’t want to use FreeNAS as that’s boring and also the whole “For ZFS get 10x your TB as RAM”-thing really scared me off anyways. So i went with mdadm raid5. Works prefectly fine on a potato cpu and 4gb RAM. It’s technically my Dad’s NAS though.

Some day i’ll build my own NAS and try all these things (BSD or Linux + ZFS). Though im not so sure about BSD, never really used it in general.

1 Like
32

It’s 1GiB for each TiB of space. This too, is just a recommendation to yield the best performance. But it will still work with less.

1 Like
33

I know. I was exaggerating on purpose. I read the thread after all. Still, people generally tell you it needs a lot of RAM.

1 Like
34

I agree. Need is a strong word. Recommended is a much better relative word.

1 Like
35

As far as I know the 1gb per tb thing is if you want to have all the metadata in memory, which would improve performance but on a home network you’re not likely to be accessing that much of the array that often.

More ram is more better but you’ll reach a point of diminishing returns pretty quickly on a home network with few users. How much ram you need really depends on your workload and the kind of data you’re storing.

1 Like
36

Keep in mind that this rule is only a requirement if you’re doing deduplication (not recommended).

If you’re not using dedup, you’re going to be fine with a small amount of RAM. I recommend figuring out how much you need for all your applications, adding 4GB for ZFS and buying the amount that makes the most sense from the number you came to.

IE: if you arrive at 33GB, there’s no sense in buying 64GB, just go out and get 32.

1 Like
37

tnx everyone . i will let you know guys how its going , at Q1 of April once i get old workstation

1 Like