Ive been doing some research and I’ve discovered IBM FLashSystem and other “enterprise” storage solutions. Do these system do what ZFS can do? Or is ZFS truly unique when it comes to bitrot protection?
ZFS is a cow.
There are other cows.
This is not the only one.
But it is nifty.
2 Likes
I use DSM on a Synology and it’s just fine
The discussion around best is similar to arguing whether MacOS vs Windows vs Fedora vs Debian etc
Ok but what about IBM FlashStorage? Does it have what ZFS has?
1 Like
Hmm…
I’m clearly too old and have spent too much time on forums for this to be my reaction.
But to answer the question, I have no idea.
“The best there is” is a really really subjective statement.
ZFS is a really solid extremely well-rounded file system and volume manager. To ask this question, you really need to understand that those are distinctly different jobs and ZFS does both. There really isn’t another option that performs both of those tasks so well.
But that doesn’t mean there aren’t alternatives. In the SMB and mid-size space, Synology as mentioned by @aj0413 is neat. BTRFS tried to do what ZFS does, where it is both a volume manager and a filesystem. BTRFS tried to re-create the wheel because of license issues related to Sun/Oracle and ZFS. But going against the billion dollar file system was no small feat.
BTRFS is a fantastic file system, but it’s not a fantastic volume manager. It literally cannot do RAID 5/6 and be stable enough to ensure your data is not corrupted. What Synology did was using mdadm (md for short) to do the RAID and overlayed BTRFS on top. This is a good design and is close enough to ZFS, IMO, to be compared directly to it in that low to mid market space.
I’m not familiar with IBM FlashStorage, so I can’t speak to that directly. I can speak to PURE Storage and NetApp/Lenovo Santricity, though. Higher end SANs like this fascinate me. Both of which do some things better than ZFS.
Santricity (NetApp E-Series, Lenovo DE-Series) is a “budget” SAN offering. It is dumb block storage and has no file system, it is used almost exclusively for connecting a hypervisor like VMWare to storage.
There are some reasons why ZFS is better than this design. They both are COW, and therefore can both do pretty cool things with snapshotting. But ZFS natively can do in-line compression and de-duplication. SANTricity cannot.
SANTricity, however, can do stuff that ZFS still can’t do. In SANTricity, there’s a concept called Dynamic Disk Pools. In ZFS, we have rigid structures whereby a “Pool” is made up by “VDEVs”. There is no such concept in SANTricity. You simply tell it how many disks in your pool you want to save for parity, and it does the work for you. Add another 12 drives? No problem. Your pool just dynamically grows. Want to change how much parity you have? No problem if you have the space!
ZFS can’t do that yet. Things like DRAID and RAID-Z expansion may eventually meld together to make it possible, but that’s at least 5 years out from being mainstream if they devs keep up the pace they are at.
SANTricity also uses mostly standards-based hardware. You can reuse the disks and shelves on a TrueNAS system if you get sick of paying NetAPP maintenance fees.
PURE storage is even better than both. It does all of those things, plus better and faster compression, deduplication and encryption than ZFS. Plus, when we are talking about cutting edge all NVME based servers running ZFS, it basically sucks right now. ZFS can’t touch PURE in terms of performance at scale. But that comes at a high cost. These systems are wholly proprietary, support contracts cost a fortune and you can’t even patch your system without one. In fact you can’t even patch your system at all, PURE engineers do it for you.
ZFS runs on basic commodity hardware. No special sauce, it will work on anything. You can have a support contract with IX, you can buy a system with IX. Or you can build your own system entirely using off the shelf components. Or even re-use cast-off enterprise gear. It doesn’t matter. It just works.
Hope that helps…
7 Likes
Thank you so much for this reply! Can PURE storage do bitrot protection like ZFS?
Yeah, all of the big “high end” storage vendors will have some sort of built in “scrubbing” mechanism like that.
What about something like the Lenovo D1224 Direct Attached Storage? (I guess I need to research the differences between DAS/SAN)
A DAS is just a fancy word for a chassis that has a bunch of hard drives in it. You plug it into a computer, a server, or a SAN via special SAS cables. Then, whatever you plug it into just sees a bunch of hard drives.
You can then put a filesystem on it like ZFS, or even NTFS or ReFS inside of StorageSpaces on Windows. Think about it like a bunch of SATA cables going into an external enclosure, but fancy. There’s no brain. Whatever you plug it into has the brain.
2 Likes
The question “What is the best X” is generally nonsensical. For most people, a cheap, slow, and reliable economy vehicle is going to be FAR superior than a cutting edge F1 racecar, because they don’t actually need the thing that the racecar can provide better than anything else, and there are a lot of things the racecar can’t do that regular people need. Like be road legal for one.
In order to evaluate between filesystems and storage device managers, you must first define your requirements for mandatory and nice-to-have functionality and how important they are to you.
Otherwise no one can ever answer your question, because you don’t know what your question even is.
For me broadly speaking, data safety is my most important need. So when ZFS sucks an egg when it comes to getting all it can from NVMe devices, it’s simply not a factor as it’s “good enough”. I certainly look forward to improvements coming, and if it was ridiculously slow and throttled everything to 20MB/s sure I’d look for greener pastures, but performance is low on the list of what I want for my storage. Likewise, different businesses have vastly different priorities that can make ZFS a perfect fit from heaven or a absolutely laughable no-go.
3 Likes
So a SAN is what I am after? Do you have any experience with something like the ThinkSystem DE2000H 2U24 SFF Hybrid Flash Array?
My question is mainly about bitrot protection. I was under the impression that only ZFS did bitrot protection. I wasnt aware of enterprise options until recently.
FWIW, I use ZFS at home and at work. Much like you are getting at, there are use cases and tiers of storage where one thing makes more sense than another. OP seems to be at a different part of the IT path than you are I, and I hope I wasn’t oversimplifying my response.
1 Like
I’ve have one on order since June (at work lol), but I haven’t gotten it yet. I have two of the 12 bay spinning-drive version of that SAN and I love them for what they do. That is a Santricty SAN.
For the record, and this one is @Log the only reason those two SANs aren’t ZFS is because they were FAR cheaper than an IXSystems box. Economies of scale and whatnot.
But other than trying to understand how all of these technologies work, I’m not sure how else we can help if you don’t explain what you are trying to do. Unless you are just trying to understand.
BTRFS also has check-summing and repair via proper parity,
An up and coming linux filesystem (that I personally look forward to) with the same is https://bcachefs.org/
Windows has ReFS, as an experimental filesystem that can do this as well, but there are definitely usability issues and I would recommend staying away. ZFS is actually beginning work on integrating the Windows compatibility project that’s been going on for a while now. Not done, but that’s what I also look forward to.
I am unsure about the various enterprise filesystems like NickKF listed.
Unless you are just trying to understand.
Good point, my response may have been unintentionally dour. Trying to figure out what one doesn’t know and asking bad questions is absolutely fair, and certainly better than not asking.
I think I also didn’t read the “Or is ZFS truly unique when it comes to bitrot protection?” in the OP so that’s on me as well.
That’s interesting. Error handling is handled at all layers of the stack in an enterprise SAN. Working outside-in, Fibre channel natively looks for errors at the equivalent of layer 1 thru 4 of the OSI model. So does iSCSI and SCSI (SAS) itself. So when a write is committed to a disk over to a SAN, there are multiple pieces of the system that are checking to ensure the data sent is the data received before it even gets written to the disk.
SATA can’t do that, and so stuff like Synology or lower-end ZFS boxes have to do it things higher up in the software stack. Bit-rot, check-summing and that stuff is similar. A misbehaving SAS drive (mostly) just dies, where a misbehaving SATA drive will keep trying to give you data even if it may be wrong.
Enterprise SANs do stuff further up the stack too, an ONTAP box from NetApp will do more to ensure data integrity than a Santricity one, for sure. But that’s part of the segmentation.
Pure is different from all of them because they don’t even have controllers on their SSDs…it’s straight flash into the PCI-E bus and all fancy proprietary software from there…
2 Likes
Hmm, so what you are saying is that the enterprise stuff does what ZFS does, only they do it in the hardware?
That’s certainly part of it, yes. But that does not mean that they also don’t have software mechanisms to do stuff too. Pure as I have said, is all software, and all proprietary.