Using LTO Tapes to backup your TrueNAS

Using LTO Tapes to backup your TrueNAS

NickF

Guru
Joined
Jun 12, 2014
Messages
763
NickF submitted a new resource:

Using LTO Tapes to backup your TrueNAS - Why data ownership trumps the cloud

Preface:

I typically start these types of articles with a definition of the title and some explanation.

Today I am going to break that trend, and jump into an anecdote from my professional life. Just about 5 years ago, I started as the Datacenter Operations Manager for a large public school district. As part of my "orientation" my director went through several key systems, and one of them was our backup strategy. At that time, we had two datacenters, each with warm backups of the...

Read more about this resource...
 

samarium

Contributor
Joined
Apr 8, 2023
Messages
192
I don't understand why you used ``dd`` to extract the tar file, then ``tar`` on the file, rather than just ``tar xf /dev/st0`` ?

Brings back memories of all the flakey tape devices I've used over the years, and the backup software, but no need to diverge into the fine details.

One thing I don't like about tape devices is the mechanical nature and mechanical failure and tape jams, especially with autoloaders. Hopefully they are much better than DLT autoloaders I had to put up with, seems like we almost always had one broken.

Good to see that prices have come down a bit, I always imagine a reasonable tape drive will be at least $5000. But I suppose for 100+TB pools you would need more recent more expensive more recent tape drive to have the capacity.

2.5TB / tape seems reasonable for low volume essentials backup, so a reasonable trade off, much better than blueray. Hopefully the drives have enough availability that you can get another on the market if the house burns down, although you will probably have higher priorities : - )

"nothing beats the bandwidth of a truck load of magtapes"
 
Last edited:

gdreade

Dabbler
Joined
Mar 11, 2015
Messages
34
(Talking to future readers here, not necessarily samarium ...)

Good to see that prices have come down a bit, I always imagine a reasonable tape drive will be at least $5000.
The prices mentioned in the article have got to be for second-hand equipment. New drives for current-generation LTO still hover around USD 5000, and a new low end autoloader or library will cost at least that much again. If you don't have one already, you're also looking at a SAS HBA for somewhere around USD 600, plus associated cabling, etc. If you're thinking about starting with a single drive and maybe later putting it in a library/autoloader as an upgrade, be aware that it appears that often a standalone drive cannot be put into a library as while the internal mechanism may be the same, the drive chassis are different between the two cases (AFAICT).

The above doesn't alter the fact that tape can still be the right solution, it just changes the calculus of your break-even point if you want to avoid used equipment.

Various people in other threads have commented about offline backup solutions where you can attach a large external disk via USB (or whatever), or even an external chassis containing multiple disks, striped. They then take a snapshot and copy it over to the removable media. That gets far more complicated when your data sets are larger than the native capacity of whatever removable disk media you have available. This is also another case where the argument for tape is stronger; even the most basic tooling for tapes has had, for decades, the capability to span a backup across however many tapes may be needed.

Hopefully the drives have enough availability that you can get another on the market if the house burns down, although you will probably have higher priorities : - )
This brings up an important point for anyone contemplating LTO: You have to be aware of the inherent migration/compatibility rules. As described in the referenced wikipedia article, a drive can write to the current and previous generation, and read from the current and last two generations.

So if you're buying used equipment and get a good deal on an LTO-5 drive, you might want to get at least one spare drive and ensure that it works, too, then store it with your offsites. If the house burns down and all your offsites are written with LTO-5, and you can no longer purchase LTO-5, -6, or -7 drives, then you won't be able to read those pristine offsite backups anyway.

FWIW, this isn't specific to tape; it would be comparable to having a backup solution that is based on Zip drives (or floppies). If you lose your only Zip drive in the fire and can't buy another (or buy a computer that can talk to a Zip drive ...), you'd be similarily hooped.

If your backup strategy is not just the usual rolling 3-2-1 but also includes long term archives, it should also include plans to migrate the archival data to newer LTO generations as your equipment gets upgraded through the years.

"nothing beats the bandwidth of a truck load of magtapes"
And related to that one, those contemplating cloud-based backup need to consider not just the cost of retrieving a backup, but how long it's going to take over the network (your cloud backup service may also have the option of shipping you physical media as a premium service).

And thanks for the Tanenbaum (mis)quote :)
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
One thing I don't like about tape devices is the mechanical nature and mechanical failure and tape jams
A year and a few days ago, I was carefully oiling and massaging the drive head bearings on a pair of HP LTO3 drives, while Abe Simpsoning the hell out of local school kids on a work experience thing. That came about because the drives were left sitting (right next to the ocean, even indoors with air conditioning is not good enough to keep things in good shape) for a few years, because they looked faulty. Turns out they weren't, the tape library just had crap PSUs (one outright dead) or some additional fault that made the drives stop working when a tape was inserted.
Of course, the order of operations was:
  1. Try tape library with drives
  2. Inspect tape drives
  3. Figure out the heads were completely seized
  4. Acquire vaguely-suitable lubricant
  5. Sit there for hours getting all six bearings to rotate freely
  6. Reassemble drive
  7. Drive doesn't work
  8. Repeat 4-6 on second drive
  9. Break ferrite shaft used to encode the head position
  10. Curse profusely
  11. Bodge a setup to run the drives without the library
  12. Holy crap the drive actually works, quick, dump everything!
  13. ...Why are 20/30 tapes empty?
So yeah, keep in mind that these things are very much not the set-it-and-forget-it sort of thing.
 

NickF

Guru
Joined
Jun 12, 2014
Messages
763
@Ericloewe is right about things not always being set and forget. I was talking to a colleague who’s a R.F. engineer recently about spares and warranties and all that jazz. The same kinda thing applies here and with my IT experience.

There are folks who have bagillions of dollars in real operating budget, some others at least get big dumps of grant money even tho they have no operating budget. Then there’s the SMB/home market people who pick up the scraps. Kinda like gaming market benefits from engineering on ai or compute focused enterprise cards.

All (most?) of these big tech companies engineer high margin enterprise components. This is a primary objective. Think a 20,000 dollar server or SAN or Switch. The BOM of these parts are usually disproportionally lower than their retail value when compared to other market segments of similar technology.

In fairness, companies also design for low power devices like phones, even though this is a lower msrgin it’s a higher volume market. Think a tv streaming set top box or a cell phone.

Tape drives are in the former category not the later.

As mentioned above, Zip disks or usb hard drives all that jazz are in the later not the former.

Because tape libraries were designed as modular devices the idea is you keep spare parts. In enterprise land, you typically have multiple fully assembled devices with service contracts. This is expensive but so are on prem SANs. The margin on these kind of devices from both the VARs and the manufacturers are rather high, and even the suppliers can be costly because these are all custom designs.

The scrappy way to do is to keep a primary production one on contract and have a generation (or two, see @gdreade ) older one. This way you can at least always read your old backups and ideally even write to them

In homelab or even scrappier land, you pick up some spare drives for your librery(s) for what shit goes sideways.

The cloud is there to make all this shit easier so you don’t think. But you pay for that privilege and they can hold your data hostage in a whole bunch of really crummy evil ways.

I was previously an infrastructure manager for a multimillion dollar public organization, but relied exclusively on grant buckets of money with strings attached. My operating budget and my staffing levels were garbage. I literally couldn’t afford to buy patch cables but I had millions of dollars in Cisco switches and WiFi APs.

Tape drives made more economical sense for me, because I could buy the spares I needed to survive. If I couldn’t pay my iron mountain bill I could recall my tapes. On the flip side, if I couldn’t pay my cloud hosting fee I was screwed because I couldn’t afford the egress fee without fighting with the CFO.

this same business model exists in other market segments for different reasons. In my homelab life, the cost of backing up my really important home videos and pictures and other stuff matters to me. The quality of those backups also matters to me. I have it on OneDrive, my TrueNAS(s), and I have a little server at a CoLo pretty far from me geographically.

But I wanted an archival copy. One that is easily accessible and potentially faster to restore than other backups for the most critical or urgent needs. No different than when I worked, we had different sets with different retentions and all that jazz,
 
Last edited:
Top