ZFS Snapshot Backup

Software & Operating Systems Linux
, ,
1

Hey all,

So I was looking for some recommendations for backing up ZFS. Currently I have it set up on Ubuntu, and have it shared through Samba. My daily driver is a windows 10 machine.

So; I have a backup set up to run each day, which backs it all up to my windows PC. I use Cobian Backup to run a differential backup; however the problem is that it takes a long time to find the differences.

Is there any software I can use to do this in reverse? Share my backup drive in windows and Ubuntu will detect it is available and push only the differences (Hopefully faster - it knowing what the differences already are (Snapshot related?))

Cheers in advance

2

This doesn’t really sound like a ZFS related problem.

Snapshots are a low level function that freezes the filesystem at a point in time, then what you do with the data is up to you. If you have a busy database that’s constantly getting changes, you can snapshot it, and then make a backup of the snapshot. ZFS automatically tracks the differences between snapshots and the “live” filesystem, and that’s what will cost you space, as only differences get recorded.

You can use ZFS snapshots for backups by making a snapshot and then sending it over the network to another zfs pool, then you can remove the snapshot on the host system. That wouldn’t really work for backing up a linux machine to a windows machine.

I don’t know your Linux box, but I’m going to hazard a guess using ZFS snapshots is overkill for backing it up.

Check out this page, it might help you:
https://help.ubuntu.com/community/BackupYourSystem/TAR

If you only want to archive files since the last time you did a backup TAR has options for that
https://www.gnu.org/software/tar/manual/html_section/tar_52.html

Making your windows drive available to the linux machine, you can use the following, change the obvious bits as needed

sudo mount -t cifs -o username=somebody //servername/share /mnt/mountpoint
3

edit I completely misunderstood the question, I just realized you’re going windows->sambashare(on ZFS)

Make use of ZFS snapshots and send/recv, it’s fast and you don’t throw away the benefits of using ZFS.

Either of the following can help you do this automatically
https://github.com/jimsalterjrs/sanoid
https://github.com/yboetz/pyznap

You can even compare snapshots with zfs diff and see what changed, though this can be slow.

On other file systems, rsync and rsnapshot are generally fast at updating.

I realize these things can take some effort to learn, but it’s worth it because then you can automate things. Just take good notes of what you do.

4

Any solutions that you can suggest? I am now on Debian following a stream of windows farces; so it would be backing up Ubuntu ZFS > Debian.

I’ve read over the sending of ZFS snapshots, but like you mentioned. It does seems a little OTT. I currently it NFS shared across.

Is there any clever software which will know only the modified files (In windows the problem was that the differential backup would spend hours checking if each file had changed - Obviously ZFS tracks that and I have the snapshots)

Any help appreciated, many thanks

5

I’ve used a program called unison in the past for backups and syncs. It can even deal with bidirectional changes (though not perfectly)

I tracks internal state, and in most cases after the initial sync is fairly fast and subsequent runs use file metadata on each machine to tell weather or not it needs to look more closely at any directory or file.

https://www.cis.upenn.edu/~bcpierce/unison/

Though I’m not sure this is what your looking for. It will only send/update the differences, but only versions/snapshots are not preserved. You’d have so use a different mechanism such as hard links or shadow copy to implement multiple dated backups without wasting a lot of space.

6

Just the live data is fine for backup so it seems like this does exactly what I want. I have it running through the initial sync now.

I’m guessing you set this to run on a cron job or something?

7

If the data direction is always one way, then you can run it as a script. (and there is a command line option to do so it won’t try to interact).

However occasionally when you are doing bidirectional sync it has to ask you for the right action. And this was the used case I had, so I would run it manually every so often.

8

I’m only interested in one way, so that is perfect. Hopefully this is what I’m looking for, will keep you updated.

It does seem to have slowed down while checking the bigger files for changes, but we will see!

9

Not too impressed at this point. 16 hours and it’s only moved 1tb. Any other suggestions?

10

Just for future use; I am now using ‘rsync’ which over NFS seems to work phenomenally well; and takes perhaps a minute or so to detect the changes across 6TB of ZFS.

Transfer speeds as you’d expect on your NFS connection