I think must be doing somthing wrong… I have had two Seagate IronWolf drives fail this MONTH and both fail with simular errors. I should gave some context first. I am using these drives in a NAS / Media center use case and they are not in any RAID configuration. I have elected to make dedicated duplicates of the drives every month or so, and keep these in a separate location. The first drive I had fail occurred after a storm that resulted in repeated power loss that corrupted the file-system, ext4, beyond repair. The second one I just had fail after attempting to make a backup of the drive. This failure is more serious as the drive fails its SMART test. But I am still able to read contents with the testdisk program. My question is, am I doing anything wrong with the treatment of these drives that would cause frequent failures? Is EXT4 not the best choice for a file system? Also could these drives be failing more due to the os, Libreelec, never putting these drives to sleep?
I have NEVER had these many drives fail within this short of a time span and are IronWolf drives just not that good. I have a few WD blue drives that are over 5 years old and still working fine.
This is likely the cause of both disk failures. But disks do die, sometimes for no reason. I had 8 WD greens all die on me within a month of each other and that was with a UPS and everything.
well it looks like the lower capacity seagate desktop drives don’t fair to well, but I am haven’t found a statistic for my drives yet ST2000VN004. So far testdisk has been able to copy 154K files with no errors from this drive. This is why I am also skeptical of the ext4 FS that I am using.
Btrfs might be a better choice as it has redundant metadata and uses copy on write so it should be more resistant to corruption from power interruption. But there’s nothing inherently wrong with ext4 compared to similar files systems.
If you want to protect from bad power, and don’t have an UPS, make sure your second copy drive is disconnected / isolated from the mains, or is far enough away that you aren’t on the same grid. This means connection is a manual task, but sounds like you are doing manual sync anyway. Wasn’t clear from OP if second location was a connected to mains location.
I am indeed already using that method of backup and it is working, but I DONT like how much I am relying on these backups to keep my data… and like I said the data recovery program “testdisk” is having no issues collecting most of the files. It just seems that my file system gets messed up and cant mount. I am guessing this is due to the “unclean” power cycling.
ext4 is a journaling file system so in the case of a power cycle reboot, should recover OK, however that isn’t what you seem to be seeing.
Even if you have a UPS, repeated power loss might still be a problem unless you have a good UPS. There are plenty of cheap crappy UPSs out there, especially when you want to control it from linux.
[Just noticed you said NAS, so more interesting file systems like btrfs and zfs might not be available]
btrfs does indeed look interesting, and the checksums and the COW may give you more protection. File systems with block checksumming give you more confidence you are getting out what you put in, or at least if it is corrupt. Something like par2 parity for forward error correction would give you more chance to fix issues since you are only using single disk, and btrfs wouldn’t be mirrored so couldn’t autocorrect by looking at the mirror disk. zfs has similar features, not sure the exact feature sets between the two, I’m using zfs YMMV.
While disks have ECC to reconstruct data blocks on read, you still hear of bitrot of data sitting on disks. If you were using btrfs or zfs then you should have more confidence that you read what you wrote, and if there were checksum errors. Both btrfs and zfs seem to support the concept of scrub where the file systems data is all read and corrected where possible if there are checksum errors, but this depends on the file system having a second copy of the data somewhere.
Maybe you need to go to 3 disks, and make the third one a different brand, which will undoubtedly be more expensive, and then update the offline copies alternatively. Sounds like you need more confidence in your backup system than it is delivering at the moment. If you use par2 you would also be able to parity check the data on the disks, although if ext4 the file system metadata would still be vulnerable.
Maybe you need to consider some online backup too, maybe encrypted before backup?
yah i have considered upgrading my MEGA account and using that for a third backup just didn’t want to spend 40$ a month for the space I needed. I am planning to separate the media centre and the NAS bc I think the louder bass from the tv area could be harming the drives lifespan. I am also going to give btrfs a shot again as the last time i tried to use it was over three years ago and it was NOT very stable. Also going to pick up a Toshiba drive as I did some research on the failure rates of different brands of drives and they seem to be consistently the least failing drives.
Not sure what NAS you have, but IIRC SYnology and QNAP both have backup options to online storage like backblaze (multiple types) and other online storage types, onedrive, gdrive, dropbox, box, carbonite, rsync.net etc. Worth checking them out you might be able to find something bundled if, ie, you have office 365 i think it includes some onedrive space, gdrive includes 15G? Hopefully you can find something cheaper than $40/mon. Maybe you only need to do a partial backup online of essential documents, rather than your full media library.
Good thought on the bass, might need to use a SSD there.
yah I might go to solid state in the future and also backblaze looks REALLY promising. And my “NAS” is just a mid tower pc with some cooled hot swap-able trays in the front. Nothing fancy for this poor college student lol. Gonna try using an Rpi today for the media center and see if my network can just stream the content via a SAMBA or NFS share. If so the NAS will get moved to less, how shal i say it, less vibration prone location lol.
What is your other hardware, such as motherboard? Some ryzen motherboards have an “aggressive sata sleep” setting on by default that can also cause problems, but can be turned off in bios.
If you are having constant power outages, you need a UPS. I recommend ones that advertise better simulated sine waves which are compatible with high efficiency “active PFC” power supplies.
This is what I personally use: https://www.amazon.com/gp/product/B00429N19W
All UPS’s will eventually need the battery replaced, so do keep in mind they are very low maintenance, but not maintenance free.
Homebrew NAS, the most flexible, and the most educational. RPi isn’t the fastest media center, but RPi2 works for me with kodi. My storage is on the server in the other room connected via ethernet and NFS.
Certainly if you are doing homebrew and ext4 then btrfs or zfs are easily doable. Using zfs I replicate from internal storage to backup storage on a different box using snapshots, and I only have to replicate the snapshots, your manual sync method would be two commands. Don’t have to scan everything like using rsync. btrfs has snapshots, but I don’t know if they can be replicated.
A further point on @Log comment about UPS. UPS manufacturers, especially of cheap UPSs, want you to buy another one when the batteries die, so make sure you get one that you can easily replace the batteries. I replaced mine about 5 years ago, and it was a PITA, and I’m thinking about how to replace them this time because they are getting old, or maybe just use 2 big sealed car batteries.
@samarium
Yah the Rpi 3 that i have works pretty good for kodi it does struggle with H265 video as it doe not have a HW decoder.
As for the UPS I have a fair amount of experience in electrical engineering an I am considering just making my own full time UPS with some 50Ah LFP battries and an inverter. it actully would end up costing about the same as a prebuilt UPS with much better specs. If I make it I will post a thread here on level1.
@Log
My NAS uses an old Athlon X4 860K on an A88X Motherboard. 16GB of Corsair Ram and a cheap ATI HD 5xxx low profile card for video acceleration. I am also using an rosewill photon 650W PSU. Basically the a Frankenstein PC that a built from old pc parts. For OS I am just using a x64 build of Libreelec as it was simple to set up and at the time had all the features I need. If i move the NAS I will probably put Linux Mint on it as this is what i am used to unless you think another flavor would be better. I was thinking arch or centOS but I am a noob when it comes to those systems.
I REALLY need to get the NAS on a good UPS just haven’t had the spare money.
For OS I am just using a x64 build of Libreelec as it was simple to set up and at the time had all the features I need. If i move the NAS I will probably put Linux Mint on it as this is what i am used to unless you think another flavor would be better. I was thinking arch or centOS but I am a noob when it comes to those systems.