I am looking for a backup solution that involves: 1) mirroring external HDDs to each other (data is mostly media files), 2) backing up workstations on shutdown to NAS storage. No RAID setup and all disk except the NAS storage is offline when not used. All systems are Linux. Previously using rsync on XFS (performant filesystem), but there’s no checksumming and file renames get treated as new files so data unnecessarily synced again.
Currently use Btrfs for its checksumming alone (to prevent silent errors which could propagate to backups and render them useless). Besides that, Btrfs shares what appears to be same features as those by backup software: snapshots, incremental backups, de-duplication, compression. Encryption requires Btrfs on LUKS while it’s builtin with borg and similar software (I’ll probably use kopia for its multithread support).
How do they compare given the similarities? I would think these features at the filesystem level might give Btrfs the edge while a backup software is filesystem-agnostic (which is normally a reason for others to use it to back up to third-party cloud). How does Btrfs on LUKS compare to borg in terms of performance? I assume the backups would still need to be on a checksum-supported filesystem (so btrfs on all drives?).
One advantage of borg is that you can exclude files by pattern (I’m not sure how you can get this with btrfs). Apparently deduplication works better than with Btrfs (not sure how or why).
Btrfs’s snapshots are atomic, which I guess is useful if backing up a live system for consistent state.
What else should know about to make a more informed decision?
Yeah, you don’t really want stuff changing while the backup is running. Snapshots solve this problem, having a point in time state.
And with (incremental) snapshots, you can run this every minute if you want to. BTRFS is super fast, even more so with incremental snapshots…just the first replication needs some time, after that, it’s just some background job.
On a block level. Which is why CoW filesystems are so fast at snapshot replication. Copying and checking/validating files is just tedious and slow work.
There is nothing faster than BTRFS/ZFS (or CoW in general) snapshot replication for backup.
they aren’t reading or writing to the snapshot. The snapshot is atomic and you replicate this snapshot, which is read-only. Life goes on and changes happen on the FS, but the snapshot is always the same.
imagine a snapshot being a backup file “outside” the filesystem. You are just “copying” that (read-only)backup file to your proper backup drive. It doesn’t interfere with anything happening in terms of applications doing writes.
This “backup file” is atomic, it just checks which blocks in the FS have a birthtime up to that very second you create the snapshot. It’s just reading some git-like tree and flipping the light switch after that and done.
Some applications could leave data in an inconsistent state if some changes were already committed but others still pending, at the moment of the snapshot. Exactly because they don’t write to the snapshot.
No. That’s not the job of storage.
If there is a need for applications to shutdown before the snapshot (I never had trouble with this and I made like 80k snapshots), kill all stuff and then do the snapshot. But nobody does that, because it doesn’t matter in practice.
The application certainly knows what has been written and what hasn’t been written. If BTRFS or ZFS returns the write to the OS, the app knows. If it doesn’t, then it didn’t happen. There may be some bytes on the disk, but if the write isn’t fully complete, the CoW FS doesn’t commit the transaction and “it never happened”.
Not per se, but it would depend on how the application actually writes (sync or not, buffered or not). In general there shouldn’t be issues, just with databases I’d take care to either use built in tools to dump/backup the state or shut them down before taking a snapshot and restarting.
A big challenge with inconsistent backups is that they look fine, until you restore (which may also look fine!) but then the users complain or the app raises an exception. The vast majority of backups aren’t used and commonly there’s only minor time windows where the app’s on disk state is inconsistent during operations involving writing so the problems can stay hidden for a very long time.
It definitely does matter in practice, and people do need to pay attention to consistency when they’re backing up with borg or btrfs. Worst case you’ll restore and it’ll be corrupt, for other things you might just cause the restore to be slower as the thing has to do some unclean recovery tasks .