The next video will demonstrate that "scrubbing" is not possible with md. When you have conflicting data, there is no mechanism to know which data is good and which data is bad.
1 Like
Ok thanks for preventing me from falling into that hole, so I'm looking into ZFS now and I cannot grasp how this functions, could you point me towards a step by step guide as to how I should setup a 2 drive, ZFS array?
.@wendell I am still confused on how you can have a zfs file system on a Freenas box for example and turn on samba protocol, then share it to a windows environment when windows uses NTFS?? Could somebody explain?
I understand that this is meant for enterprise users, but is it overkill for consumers?
I'm currently using a Bay Trail computer running windows as a media server. I keep all of my files on a 6TB NTFS drive and backup to a 6TB external drive every once in a while. I also use a data scrubbing program called SpinRite once a year to combat bitrot.
Is my current setup fine for home users that don't have enterprise worries? (like 100% uptime)
Its a bit over my lvl right now to be honest but i get a lot of bits and pieces.
And i like to learn things the right way upfront not oversimplified. So its fine for me.
Your explanations and graphics are very helpful to me.
My next machine will have ECC-RAM is my big takeaway.
I hate corrupt files and already have a Pro-SSD because of that.
Look forward to the next part.
I was just wondering the same thing, although at the rate he was receiving feedback I also assumed it was network banter or a IRC chat in command line lol.
Doesn't matter. Redundancy info is stored cross device. So a failing checksum triggers a sanity Che k across all data for a given block. Other drives checksum will pass.
Chances of a good checksum ablgainst bad data being accidentally generated due to multiple failures is like 2 quadrillion to one
1 Like
This is all a bit too esoteric for me but MAN do I want to learn. I'm learning so much about computers (a little late, as I'm a total noob at 27 years old) but I'm falling in love with computers for the first time and I want to learn so much. Thank you, @Wendell for sharing your knowledge with us.
1 Like
Never to late to do anything :D
1 Like
Thanks for the info I learn alot.
Wendell knows how to terrify people :)
Cant wait for more....So interesting.
Thanks for the video Wendell, very interesting stuff. Have you ever played with ZFS on linux, and if so what are your thoughts on it?
2 Likes
What if you're stuck with windows? Is ReFS a good choice if a NAS / SAN is not an option?
Very good video!!!
It is true that a hardware RAID is a technical solution that is part of a technical solution, and doesn't solve all issues. However, it's still a very practical tool that economically makes a lot of sense in my opinion. A good hardware RAID controller with good quality SAS drives, reduces the chances of common data corruption as such, and provides a good base performance. The implementation of a complete data security solution, is of course not possible by just buying a RAID controller and some SAS drives. The price of good quality hardware only makes sense for enterprise level operations in my opinion. So does a SAN in many cases. Also, for some applications, more flexibility is required, that can only be had with software solutions. ZFS is the standard storage system on many modern linux distros, and it's still the preferred solution in my opinion for most users on all levels that are looking for a complete solution that is pretty manageable. OpenSuSE for instance will by default use btrfs for the system, and ZFS for the data now, and that is a more universal solution than running everything on btrfs, however, btrfs is evolving quickly, and the performance has gone up quite considerably with the last versions, and I think it's a great solution for first line data integrity and security, at workstation level for instance, because it offers a lot of really handy tools that just work, and all it requires is a JBOD config of some kind, regardless of hardware basically. I use it in my home workstation for my first line data storage, because it makes the data very portable, and not just the data, but the entire system, and it's pretty reliable in that aspect. Fact is, a PCIe SATA controller is dirt cheap, and btrfs allows you to pool m.2, SATA SSD/HDD, USB, USB over TCP (very important evolution in linux is the ability of kvm to rout USB over TCP since kernel 3.18). It all depends on the setup you're after, and the constellation of the network and system. In the end, it's often about avoiding bottlenecks and weak links. ZFS deals with a lot of weak links in the storage department, btrfs deals with more than that, maybe not as good yet as ZFS in the storage department, but it's a solution that encompasses more stuff than ZFS on a general system level, even though it's not quite mature yet, the biggest problem being the fact that RAID 5/6 operation is not recommended at this time, and it may never be, who knows.
So what does a mere mortal do to keep his files safe for decades?
@wendell Nice video - very informative.
Now to plan a proper storage build - instead of just the Synology NAS...
In the mean time, I wonder if you (or someone else) may be able to advise: Impact VII motherboard + 2x WD Red Pro drives... was going to RAID 1 them as local storage for the PC. Still a good idea, or would you go a different route? Windows 7, SSD as boot drive, Optical drive installed, so no spare SATA ports.
Cheers.
just remember ZFS started on Solaris so it totally works there too.
Thanks Wendell. It really amazes me on the amount of knowledge you possess. You and Steve Gibson are the only people I know have the most comprehensive answer to EVERYTHING.
Wendell, what do you think about new Microsoft File System: ReFS ?
Also will you do any testing of it.
2 Likes