My laptop is full, my external drive is full, I don't have a NAS because thats money (or atleast a serious investment in time and effort to make sure a dont goof it up and loose everything)
I wouldn't describe myself as a file hoarder, but I do have a lot of stuff I currently do not require access to but may very well need in the future. We're talking tonnes of project files, family photos, video/audio files, ISOs and disk images, you get the idea
If I were to buy a brand new 4TB HDD, create a LUKS encrypted ext4 file system on it, dump everything I owned onto it, and leave it at my parents house in a dry and cool place (say in the loft, the fireproof cabinet, hell even on the dining room table), would I be able to come back to it after any of a number of possible disasters has bestowed itself upon my system and be able to trust the HDD to have retained all the data I initially put on it
Bonus question, what sort of HDD would I be looking for in this scenario? I understand the WD reds are meant to be reliable but that may be more along the lines of not wearing out from continuous access rather than longevity over a sustained period of inactivity
I don't see why it wouldn't be fine. Get something cheap, you certainly don't need a NAS disk or other kind of near line disk for something which is going to be written to once and then left in a box for a few years.
No. Best case it'll work, but worst case the drive burns etc. You also have things such as bit decay that occur. You shouldn't leave data sitting around for that long. Yearly you should update the data on the drive, at least. Definitely a fireproof/waterproof safe is best, though.
In practice I will probably be connecting this drive up about once a year to add content to it. It would be wise to keep a similar folder structure on both my external drive and the 4TB backup drive to allow for easier migration of files
This would also allow me to use the disks SMART feature to check it's still in good health
Wrong. Hard drives are vulnerable to bitrot after about 5 years without power, after 10 it's almost guaranteed several things will be corrupted. @HypnoticMicrowave use DVDs for long term storage they won't suffer from bitrot. There are DVDs made specifically for archival purposes (I know nowhere near 4Tb) use those. Anything magnetic is vulnerable to bitrot.
I would not trust a harddrive that has been unplugged for years. Magnetic fields degrade about 1% each year. I agree with @soulfallen that optical media made for the purpose is the best way to go.
That's a bold statement. Logic dictates that degradation should depend on data density, not just "any disk ever made after 5 years".
For HDD cold storage, there are two main problems. First, the bit flip (or bitrot, data rot, whatever you call it) - generally, the denser the data on disk, the easier for it to happen. It may be caused by a miniscule defect on a magnetic surface or (and I know it sounds ridiculous, but really may happen) simply by radiation - single charged particle goes through the platter and there you go, magnetic domain has changed its orientation. On FS level, people here will recommend to use zfs, on lower level there's T10-PI disks which provide checksums for every block of data. Second, lubricant degradation. It's not supposed to not be used, it will lose its properties with time, and chances are, your drive simply won't spin up 10 years later.
Optical storage (CD/DVD) is not an option. Plastic degrades with time, and surface gets clouded. Unless there is some fancy solution with non-amorphous, stable crystalline surface, I don't see it working for long-term cold storage.
I'm pretty sure magnetic tapes are still the best solution, and even then you'd need some sort of environment control.
Hey, I'm not talking about models, and you know it. But ye olden 14GB HDD won't suffer from bitrot in 5 years. These things were indestructible, and 4TB new one will start suffering from it in maybe 2 years, if not faster.
Edit: Regarding the fancy DVD+R -- eeeeh, let me stay skeptical about it, OK? Manufacturers said the very same stuff (including "stores data for 1000 years") about the very first optical media as well. Unless they want to guarantee data retention with a huge insurance check, I don't trust their "1000+ years" assurances. In fact, every time I had to ask a vendor or manufacturer if they can guarantee data safety in a written form, each of them chickened out. Storage manufacturers never really guarantee anything, they know there are variables they can't account for.
I would agree, i'm simply asserting a DVD would be more reliable as it isn't magnetic. @dmj i don't really buy the 1000 years either i'm simply pointing out it is made for the purpose of longevity and would be more reliable than a hard drive.
Depends a bit on how you handle the discs. And also the quality of the used disks, and the burner aswell. But yeah i´m personally a big a fan of DVD burners. I allways add one to my builds. I still use dvd´s very often for installing systems.
As long as the HDD is taken proper care of(low moisture, temps not too high, not too cold) it will last for more than a century, however, the data stored on there may not last for more than a few years without being maintained. What you want is a file system that attempts to prevent bit rot. The best thing I have done, IMO is to keep one on site that is running and maintained and an off site that you can bring back every 6mo or so and completely rewrite all the data. That is currently my solution and it seems to work. I also have multiple copies locally and remote.