Cost-effective long-term archiving?

(maybe HDDs is the wrong forum for this, I just took a guess)

I've got a FreeNAS box with 4 1TB hard drives in it. It works great, plenty of CPU and RAM but the server is limited to 4 hard drives since it's a 1U server. It's also in my bedroom so I would rather not do anything insanely noisy, which is one thing I'm worried about with doing like L1T and buying up old Fibre Channel enclosures.

I need a place to store pictures for my family. My dad is not computer-illiterate, but has very little concept of hard drives dying. He operates under the assumption that if he puts things on an external hard drive that it's just always going to be there (we have a 10 year old 1TB external hard drive that he stored our entire photo collection on before I got the NAS set up).

The problem is 2 and 3TB drives are expensive, and we rarely need to access all these photos. My dad suggested Blu-Rays, and he's thinking burn everything to Blu-Rays and stick a spindle of Blu-Rays into a safe. Sounds like a good idea in principle, but I'm worried it'll be a massive ordeal and be cost-prohibitive, and I'm especially nervous about using a proprietary-ish medium like BluRay to store so much data.

TLDR: I need an economical long-term backup system, preferably physical media that can be put into a safe.

Ideas?

I would say, the cheapest option is to use some sort of cloud backup service like crashplan. That way you just pay a subscription and you get all the benefits of enterprise level backups and redundancy without having to do it yourself.

Blu-ray discs might also be a chap option as you can get 50gb per disc and the burners are relatively cheap (compared to tape drives) but I'm not sure how long they last before they start to deteriorate. Burnt discs don't have the life of pressed discs like you get when you by a movie or something. But I'm not sure. Personally I would do both as you don't want to put all you eggs in one basket. And I'd make fresh copies of the blurays every so many years (depending on what the life of the discs is expected to be).

Long term optical disks, be they blue ray or DVD, are really the only option when talking about long term physical storage. The ones that are made to last year's upon years will have archive or similar wording denoted in the name.

2 Likes

If do archive to blu ray do it in double or preferably triplicate. Put one copy off site.

I see they make a type M disc that looks to be quite expensive and requires a compatible drive to burn as well.

Another option is to pick up a few 4TB or so USB hard drives and use them for backup on and off site. You can pick them up on sale for under $100 if you watch Slickdeals.com

1 Like

Amazon Glacier might be interesting for you. A Terabyte costs around 4$ a month.

You will be probably best served by choosing a Service now and then moving your backup to another service in the future. When storage becomes cheaper or the other service is going out of business. Something like being a digital nomad moving your backup cattle to greener pastures.

The Blue-Ray solution is really the best, but I think it's going to get expensive pretty fast if you want to do a 1TB archive. Also you should keep them in a dark and dry place to prevent them from get ruined soon. A 2TB drive that you connect one in a while to add stuff on will last as long as those Blue-Ray DVDs in my opinion but needs to be tested for a month for safety purposes because things have the tendency to fail in the short run if they have a defect.

I too have been looking into long-term backup of data for the office and this is the same conclusion I came to. Archival disks of some kind seem to be the best/cheapest option that also requires the least maintenance. Just check the disks every x number of years (every year is great if you have the time for it).

It really boils down to the ammount of data and the rate it grows at.The most effective way would be to get a fairly recent ,used LTO drive and some tapes. But that is after you decide to agree on fairly high initial investment cost.Its even more cost effective if data is compressible .If its few hundred gigs than bd drive and several discs should do.Also dont rely only on online servces, they may go down if disaster happens or a lightning hits data center .Happened before.Also bluray disc has one advantage, you cant accidentally delete or modify data once its written.(minus rewritabe discs of course)

M-disc is probably the most plug-and-play way to go if you don't want to rely on cloud services.

LTO works too but you can't store them outside in the garage or wherever if you live in a warm climate, since tape will die if it gets too hot.

Some kind of magnetic tape could also work.

Everybody has different requirements... here's my personal requirements and solution.


Requirements:

  • Home videos and pics. I add about 300GB/yr and that rate grows about 50GB/yr.
  • Encrypting is not necessary.
  • On-demand access not required.
  • Interval of archive... semi-annual
  • Not a fan of cloud storage.... expensive and you give up control of your data. (basically it's more features than I care to pay for and I have concerns over privacy)

My solution:

  • 4TB passport ultra external drives.
  • Multiple Offsite, cold backups.
  • At my current rate, this plan will be good for about 5 years. By then I expect I will change solutions to what's most viable at that time.

It's maybe not the cheapest, but it fills my requirements perfectly and these drives are extremely reliable.
For long time storage, two things are most important...

Offsite: - The most important feature in your backup plan.

"fire boxes" and safes WILL NOT protect digital storage. I did tons of research into what fire boxes can and can't withstand. If anyone is curious for a long explanation of how they work and what they can and can't do, I'll post another reply; but the long-short of it is... unless you want to spend >$10,000 they can only keep paper documents from spontaneously combusting... nothing more. Everything from blurays and spinning metal will fail in the high temperature steam in fire boxes and your data will be lost in a house fire.

The only way to protect your data from house fires is to have multiple offsite copies in trusted locations.

Cold: - The hdd's are not powered.

This extends the hard drive's life almost indefinitely at the cost of on-demand access. I can easily drive over and plug the thing into my computer if needed though. It's 10 minutes away and I only need to do that twice a year though... small cost to the life the drives will have.

Sounds like 1-2 external hard drives is the best option. If I keep them unpowered and "refresh" the data each year I think it'll work. The trick will be finding a trusted offsite location since we don't currently have a storage place or anything.

Crashplan also looks good, but I don't know if my parents will go for spending that much money.

I wondered if there were any practical tape backup options for consumers. Anybody know?

1 Like

I use Amazon Glacier. Super cheap, redundant, and items can usually be retrieved within a day if I need them.

Only the ususal "enterprise leftovers" I'm afraid. I would search "tape" on www.reddit.com/r/homelab .There are some people who are actually running tape drives for backup in their setup. I'm still reading.

If you get over the initial cost of the hardware tape is pretty good. As Andrew S. Tanenbaum once wrote: “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.”

Which is something to keep in mind if you have to completely upload your 4 Terabytes to the cloud or download them after disaster struck.

Since tapes only cost 15-20$ and can store up to 3TB (for LTO5) they are damn neat for offsite storage. LTO4 stuff only does 1.6 TB per tape but the hardware for it is cheaper.

"offsite" can be anything.

Heck, even sealing it in plastic and foil and burring it a couple feet down in the backyard is technically "offsite". It'll be safe in a house fire or lighting strike. ;)

I use fiancee's parents's house. If you don't trust the custodian to nose through it just encrypt it.

I'll throw my 2 cents in with what I use. Backblaze B2 on my synology, my parents server, and a friends that I manage. Currently have 700GB and am paying $3.

1 Like

Step 1: external hard drive.

Step 2: Cloud back up.

Tapes are crap. They are expensive, the drives go bad, the standards change... tape is the worst - especially for long term.

Blu-ray discs are the winner here, but it requires a smart plan, and burning of discs frequently. You should re-archive everything on a yearly basis and rotate out. Again, cool, dry locations, off-site, and multiple sites. The fact that you will revisit you plan in 5 years is a good idea at the rate optical media is going out of style. Even if you use another method, this is pretty cheap as a secondary method, or disaster scenario or for legacy purposes (trusted family member has a copy of your data in case you die).

The 2nd option is a cloud backup service. I'm not a fan either, but @Dexter_Kane, as usual, is right on the money. You can just backup your archive there. 300GB plus 50GB a year isn't too bad to manage with hard drives, and I don't think you'll ever get ahead of what can be served with a service of this type. Also, always keep everything on mirrored drives.

4 Likes

This looks like a fantastic option since what we need is basically just some offsite "if FreeNAS goes to the dark side, runs out of space, or gets burned in a housefire" insurance. My only question about this is, let's say you have some major catastrophe and lose all local copies of your data. Is there some way to get all your data (like can they mail you a hard drive or something?) or do you have to pay per gigabyte and wait for all 1TB of it to download?

What kind of software would you use to burn BluRays? I assume you need some kind of utility to burn a large amount of files spread across multiple disks. And will I be tied to one software/vendor here, or is multi-disk BluRay file storage a pretty standardized thing?