Lets talk backups! (share strategy, knowledge and resouces)

Software & Operating Systems Linux
,
1

I want to encourage everybody to post their strategy regarding backups along with their past experiences, useful information and software regarding this topic

That way we can discuss the up-and downsides of our specific setups, share our knowledge and learn from each other.

I Obviousely gotta start with my setup/strategy:

My backup location is a separate Harddrive in my Linux server who’s folders are shared with SAMBA using seperate users/owners to lock down write access.

The three windows laptops (brother win10, mother win7, and my win10 dualboot) will probably be backed up using windows built in software. (although I haven't tried that yet)

My laptops data partition is backed up using FreeFileSync with versioning enabled.

My laptop also runs Ubuntu Gnome which I used to back up using Timeshift rsync. But this needs physical access to the backup location so I’m still searching for a way to back up the linux system to a remote location.

I may also experiment with my Fathers MacBook. Don’t know if I really wanna deal with the Apple-headache.

The server: I am still searching for a way to frequently easily and automatically back-up the OS, Packages, Settings...
the backups don’t really have to be backed up again and any other data will somehow be backed up to an external harddrive from time to time.

If I needed it to be bulletproof I would mirror the server every month or so and store the immage at my moms work.

Cool Software:

FreeFilesync because it’s cross platform and can be nicely automated. It can be used to synchronize or update all kinds of things and has astonishing amounts of functionality and customization under the hood.

Timeshift rsync is a cool and simple linux tool to take system snapshots (by default, it doesn’t back up the home folder). It has a simple gui and a command line interface. (which I didnt get to work yet)

I would obviously also appreciate help in filling the unsolved gaps in my plan along with all general feedback although helping me should not be the focus of this thread

The background story of my reason to start this threat (read it if you want)
I‘ m currently rebuilding my Linux server (samba, owncloud and hopefully soon plex or kodi)
fileserver/owncloud server and hopefully soon media-server. (1st time working without a gui :)) At the same time I’m finally gonna put a real backup strategy for all my families into place. I’m really struggling to come up with a good plan and in finding good, understandable resources and information.
For windows there’s heaps of contradictory information floating around along with everybody who wants to sell you their software. And the few good guides for Linux I could find are all very broad and technical.
But since I highly appreciate the information sharing and having an educated discussion nature of this Forum (rather than help me/ I’m to lazy to do the research myself) I thought rather than asking for advice on my specific setup and problems I would encourage you to post your backup mechanisms along with useful information and software.
Basically I hope this becomes a place where we can share our knowledge and experience and discuss our specific systems.
(no idea if this is the right category

2

Yeah, I just posted this in networking, I wasn't sure where to put it either... Edit: when it linked to the other page it dumped the formatting it is not a wall of text as below...

3

have you looked into FreeFilesync??? it can be configured to do all kinds of syncronization otions and can be automated with realtimesync and the startup folder.

4

Because I know next to nothing about software, and everything I do know is self taught;
I take the back up hard drive and put it into my laptop. I then connect the hdd/ssd that I want a copy from to the laptops usb2 with a usb2 to ide/sata adapter and manually copy and paste onto the back up hard drive. I know this isn't efficient or maybe even correct, but it's what I can figure out. The software that comes with back up drives is like in a foreign language to me. I graduated from high school without ever seeing a computer and my life work has been in manual labor, so I haven't been exposed to much computer related stuff. Trying to learn without mentoring is hard.

5

I've looked at a few of them but I know very little about what is good and what is crap. I guess I'm looking for someone with a setup like that who has implemented it... Have you used it?

6

i can only give you the same advice: look into freefilesync! the basic functionality is pretty intuitive and easy to understand

1 Like
7

yes a lot actually but mostly one way

Currently i'm using it ro backup/update/syncronize my laptops data partition to my nas. but i also used it with my mums computer so that it automatically backs up her home folder when she plugs in a specific external harddrive
and for my brother i configured it so that it takes a snapshot of his essay every 10 minutes to a flashdrive when it is plugged in

its basic functionality is that you select two folders and compare them either by modification time, file size, or it's contents. when this is finished it shows you all the differences
then you can choose to update/sync it in a number of different ways.

I would suggest to try it out on some folder and then you can look into automating the process

8

Thanks, I'll set up a test situation and go from there.

1 Like
9

What are resouces?

10

yeah keep me up to date i have some performance issues (cap at 10mbps) but this is likely very specific to my setup
(linux/samba to windows plus freefilesync)

11

with resources i meant guides, tutorials, usefull software and all kinds of information

12

I think he's just being a dick and mocking your spelling...

4 Likes
13

I've got a lot of experience with what to do and what not to do when it comes to backups. I'll share a short version here.

First, let's talk about mistakes I made. When I was just out of high school (around 2007), my mom had a couple hard drives filled with pictures, nearly 400GB all in all. It was her hobby and she was crazy about it. She had a hard drive start to die on her. I offered to try and fix it. Mistake #1. After I realized it was having issues, I kept trying to use the drive. What I should have done was leave it (the read head was failing and damaging the platters) and let the pros swap the platters and pull the data. That didn't happen immediately and when we finally did, about 1/3 of the data was unrecoverable.

Lesson 1: Don't fuck with broken drives when it comes to data that's irreplaceable.

Lesson 2: Have backups.

This is when I set about making sure we had working backups for everything. I bought a cheap server. Some dual core celeron with 4GB ram and installed Windows Home Server on it. My first foray into servers. With 4TB of storage in 2008, I felt like a badass. Backups worked wonders. I had to restore things every now and then. While it was a bit slow, it was also extremely robust. The backup software made block-level incremental backups, which basically means only changed sectors on the disk are transferred to the server. This was good for a few years, but then our (my) storage requirements increased. I upgraded from the celeron to a Xeon 1230 (v1) in 2011 and installed freenas. This was fucking awesome. FreeNAS did everything I needed and was free. From there, I kept with freenas, but the backups software on Windows didn't support linux. Okay, what's cross platform? That's when I found Crashplan. I tried it out and liked it, so I bought a subscription to use "cloud" backup for critical data. From there, we backed up to two targets, the NAS and their offsite cloud. All was right in the world and I'm still using it today. There's a few complaints I have about crashplan, but that's a story for a different github repo.

On the server side of things, it's a bit different. Crashplan works well when your required TTL is something like 2 days. Restoring from their offsite backup works at a slow rate of 25mbps (I've got a 120/30 pipe) and that's not acceptable for enterprise. In my previous company, we used veeam for the VMWare backups. This allowed us to instantly boot a backup if we needed (the backups were stored in a form that would allow very fast restoration).

I've since moved to a company where we're using CEPH and OpenStack, so we're not really concerned about backups on the VM level. We do backup MySQL databases, fileservers and the likes, but as far as OS level goes, we're confident that CEPH won't screw it up. Ceph is a clustered storage software. It's extremely robust and we're keeping 4 copies of mission critical data and 3 copies of everything else. Basically if a disk goes offline in ceph, it instantly knows and takes the other copies of all that data that's missing and redistributes it to safe places. In our datacenter, we can lose an entire RACK (that's 500TB) and still not result in any lost data.

2 Likes
14

thanks a lot for your insight

Gonna definitely give that a try. How should i handle the server side since i'm running ubuntu server instead of freenas. will SAMBA be just fine??

and im really need a fast way to take and restore a snapshot of the core ubuntu system for when (not if :) ) i break it.

15

Have a look at either BTRFS or LVM snapshots. That's the way I'd do it.

Install crashplan on the Ubuntu server and you can use that running crashplan instance as a backup target for the client instances on machines throughout your house (and if you open ports, outside your house). Samba would be a fine option, but I think you'll get better results from the CrashPlan agent. (P.S. It can run headless and I'm working on software to help with management of headless instances)

2 Likes
16

cool keep me up to date

17

It will be on github shortly. I'm just trying to get initial functionality before I release it.

18

no never heard of it

isn't btrfs snapshots specific to btrfs?? which i dont/cant run

but lvm snapshots looks really promising at first glance (especially since i have a seperate os drive

19

Yeah, you'll have to reinstall the system, or at least take an image, reformat and restore, but it's definitely worth it.

Also, you can convert an ext4 filesystem to btrfs with btrfs-convert. This guide is for 12.10, but any newer version should be pretty much the same process.

20

My backup system is pretty simple.

I rent a VPS through Vultr, Ubuntu 16.04 server to be specific, and have Nextcloud setup on it. I have the desktop clients on my PC's and like that it has dropbox-like functionality, but I certainly feel safer using Nextcloud.