LTO Tape Suggestions

Has anyone messed with lto tape backup before? I’ve been tasked with figuring out a good solution.

I’d like to put an internal tape drive into this guy and use Linux/FeeBSD, but I don’t know what my software options are…

Isn’t this what tar was originally for?

@cotton and @MarcT, @Goalkeeper mentioned you might have some insight.

3 Likes

This is exactly what tar is built for, tho now you probs don’t need it tbh.

Look into what storage capacity you need. I think me’re on LTO8 now, and theres weird backwards compatibility swaps between LTO 6 thru 8.

But, if you just need a tape solution theres TB3 solutions nowadays that work wonders. Got mac windows and linux software. I think LTT looked at one?

2 Likes

This should help:

1 Like

Yeah, up to lto8 now. I think we’ll end up going with 7 though. It’s all very expensive, I want to make a well informed recommendation on the hardware.

I should add that the main motivation for going to tape is offline shelf life, not capacity. The need is to keep offline archival data on a format that won’t degrade after 10+ years.

Then get M Discs dummy.

1 Like

Capacity isn’t that low unfortunately. I do need something on the scale of terabytes.

I’m looking at this drive, but I can’t find any documentation on it, so that’s kind of a red flag. I reached out to the seller to see if they have more info.

There is a 30 day return policy so maybe worth a shot? I’m mainly worried that it might not be genuine IBM and could be unreliable.

1 Like

FYI there’s some sort of patent clusterfuck going on with the underlying technology that allows LTO-7 and LTO-8 tapes to even exist, despite the fact that consortiums like the LTO group are supposed to prevent this sort thing with patent sharing/FRAND agreements. LTO-6 is unaffected

Frankly even while ignoring the completely outrageous upfront costs involved with the drives, the current lowest (and optimistic) price for (out of stock) LTO-8 tapes is only barely better than shucking Easystores/WD essentials for price per TB.

Also, 30TB compressed my ass.

1 Like

Incompressible is 12TB for LTO -8 and the HP tapes are $150 from B&H.

Again, though, the multi-decade shelf life is what I’m after here… otherwise I’d stick with hard drives.

That would definitely explain the price. Maybe I should just go with LTO-6 then… 2.5TB is kind of a bummer though.

Do you have any source/reference for that (patent issues)?


Update:

Hello

This is a new sealed drive

We don’t have a Manuel

It is a regular lto8 drive

We can ship today

Todd

:man_shrugging:

Ask IBM if they know what this is:

01PL557

2 Likes

Only just seen this, but yes - I’ve quite a bit of experience with tape on Linux and Solaris, including robotic tape libraries.

First thing - has your server got an interface to connect to the tape drive? Looks like that particular drive uses SAS2, but you’ll also encounter fiberchannel tape and LVD SCSI.

Second - Tape drives are very susceptible to dust contamination. If you’re only planning occasional use (once a week, once a month) - it might be better to go for an external drive in an enclosure. This has two advantages; you can use the drive on multiple servers (provided they have an external interface port), and secondly you can turn the drive off and put it away when not in use.

Internal drives are subject to the server’s airflow all the time, and the door flap does nothing to prevent dust ingress. One of my internal LTO2 drives died last week because of this. I’m hoping to remove it, strip it down and clean it; but that requires a server outage (which an external drive would not).

Thirdly - backup application. Tar is great for occasional use, and the format is standard. However, with tar you usually have to read through the entire backup from the beginning to restore a single file. With some tricks you can log the offset of each file within the tar archive when the backup is made, then if a restore is needed use “mt seek” to skip straight to the file on tape (saves reading the whole tape to restore one file).

The Linux “dump” utility automates this logging of tape position via its “quick file access” (QFA) feature. For home use or small office, using LVM snapshots and “dump” with QFA is a powerful backup solution, but you still have to do the media management yourself (ie manage the cycle of which tape to use when).

For a larger environment, you’ll probably want a dedicated backup application and a robotic tape library with multiple drives. The tapes have a barcode attached to the rear and the tape library has a barcode reader. The application can then track each tape and knows when the backups expire and the tape can be re-used.

Finally - Most recent tape drives support “TapeAlert”, and it’s useful to know you can manually read “TapeAlerts” with the Linux “smartctl” utility, for example (from my failing, dust contaminated LTO2 drive):

# smartctl -d scsi --all /dev/sg4
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.4.172] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               CERTANCE
Product:              ULTRIUM 2
Revision:             1914
Serial number:        xxxxxxx
Device type:          tape
Local Time is:        Thu Mar 14 15:09:51 2019 GMT
Temperature Warning:  Enabled

=== START OF READ SMART DATA SECTION ===
TapeAlert Supported
TapeAlert Errors (C=Critical, W=Warning, I=Informational):
[0x0f] W: The memory in the tape cartridge has failed, which reduces
  performance. Do not use the cartridge for further write operations.
[0x11] W: You have loaded a cartridge of a type that is read-only in this drive.
  The cartridge will appear as write-protected.
[0x12] W: The tape directory on the tape cartridge has been corrupted. File
  search performance will be degraded. The tape directory can be rebuilt
  by reading all the data on the cartridge.
Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:          0        0         0         0          0          0.000           0
write:         0        0         0         0          0          0.000           0

Device does not support Self Test logging

# smartctl -d scsi --all /dev/sg4
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.4.172] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               CERTANCE
Product:              ULTRIUM 2
Revision:             1914
Serial number:        xxxxxxx
Device type:          tape
Local Time is:        Thu Mar 14 15:31:15 2019 GMT
Temperature Warning:  Enabled

=== START OF READ SMART DATA SECTION ===
TapeAlert Supported
TapeAlert Errors (C=Critical, W=Warning, I=Informational):
[0x05] C: The tape is damaged or the drive is faulty. Call the tape drive
  supplier helpline.
[0x14] C: The tape drive needs cleaning:
  1. If the operation has stopped, eject the tape and clean the drive.
  2. If the operation has not stopped, wait for it to finish and then
  clean the drive.
  Check the tape drive users manual for device specific cleaning instructions.
[0x03] W: The operation has stopped because an error has occurred while reading
  or writing data that the drive cannot correct.
Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:          0        0         0         0          0          0.000           0
write:         0        0         0         0          0          0.000           0

Device does not support Self Test logging
4 Likes

Sorry that I’m late to the party.

This post above mine is quality stuff: LTO Tape Suggestions

I have to be honest, I’m not 100% sure if you’re simply asking is LTO a good way to write to tape or if it’s a more general question about strategies for backing up infrastructure locally. I’m going to go with general advice about LTO. But, if you’d like more details or my experience, just let me know.

Without getting into the weeds too much, I’ve used LTO 4 && 6. Between two Dell Powervaults, each with two tape libraries and drives, we manage to backup 20+ terabytes (uncompressed) a week to tape. This included, in general, nightly incrementals, and weekly full backups for our entire infrastructure. For the infrastructure going to LTO-6, I’d say it accounts for about 12-16 TB. We burn through probably 8-10 tapes per week (conservative estimate).

Cotton

2 Likes

@MarcT, @cotton, thank you for your replies. Very helpful info!

My use case is to archive data to tape as it is phased out of production (sporadically throughout the work week). Once a tape fills up, it’ll be moved to offsite cold storage. I’m probably looking at ~50TB/year or so, so tape turnover will be infrequent.

One question: is it possible to browse a tape over network via a file sharing protocol? This would just be for reads.