Backups, backups, backups. (Facepalm, facepalm, facepalm.)

game_of_thrones_data_loss_meme

mostly a rant, but open to suggestions

“There are those who do backups, and those who just haven’t lost anything yet.”
This is one of the things I tell people when preaching about backups. Unfortunately it seems like my fellow humans just don’t get it. Even if they’re tech savy and see the business case, they don’t do it.

Current case: Small office (6 people) at a non-profit, with a 12 year old D-link NAS for central file storage. No backups. Did a once-over of their tech 2 months ago, wrote a nice report with an easy-to-understand summary for their board. NAS is failing, recommend immediate backup of data and replace as soon as $ allow.
Called back in this morning. NAS is being screwy and only some users can access files. Asked if they did a backup so we can try some things that might cause loss of data, replace drives, replace unit, etc. Nope. No backup.
Grab a drive and attempt a backup via the NAS’s USB port. Device GUI doesn’t support this. No SSH access either. So we plug the drive into a free desktop (wired 1gbps network connection) and start a ye olde copy-paste. About 260GB. It’s been about 5 hours and it’s done about 40GB. Sweet mother; this’ll be fun. Hopefully the NAS stays alive long enough to finish the copy.

~For those who a curious, it’s a DNS-323 with two rando consumer drives in RAID1. So we can’t just pull a drive and copy it that way. Also no way to get SSH/CLI access without breaking the config, so we can’t hack in USB either. But none of this would matter if they’d just done a backup like I told them to.

~~I should also point out that I offered to do a backup for them, and they turned me down. So it’s not lack of tech knowledge.

So. How do y’all talk to your peeps about backups? Anyone have any success stories I could learn from?

5 Likes

There are those who do “backups”, and there are those who do backups and test.

I’ve seen people in my work setup backups for projects then forget about actually maintaining them. And I’ve seen those same people start recovering from scratch because the backup process wasn’t actually backing up anything.

4 Likes

I know it doesn’t solve the problem in the end, but they’re at least still a step further than most of the people I try to help with this?.. :man_shrugging:

The thing that seems so crazy to me is that they say no even when I offer to do all the work (and in some cases even provide the resources). How can having a backup solution in place sound like a bad thing someone would want to say no to?

1 Like

Maybe they don’t understand the value of data to their company, and perhaps they think it’s just another cost. If they don’t really understand how valuable (or not) their data is they probably don’t see the value in backups.

Usually I start with “If none of your computers powered on right now, and the data couldn’t be rescued, what would you do? Right now, in this moment.” Most of the time this elicits a blank stare. So I help them along. “Your finance person is saying they need their payroll info or no one gets paid tomorrow. What’s the plan?” At this point they either get it, or they go with something like “I’m not worried about that happening / We’d just figure something out.”
Maybe my doomsday scenario is too fanciful? I wonder if I could find some stories written by non-techs on what loosing data was like, so people can hear it from a “regular person” instead of a “nerd who just want’s to spend all their money” :thinking:

1 Like

http://www.taobackup.com/

http://www.taobackup.com/wailing.cgi

3 Likes

I’ll just say from a former web developer angle, I’ve seen people complain about doing backups of sites and SQL databases so when the crap hit the fan(failure of local backups or hosting company had a failure too) it was days or weeks of unnecessary amount of “work”. Nothing really has changed since the early 2000s, some bean counters may feel downtime is the cost of doing business and backups are an unnecessary expense :thinking:

Back in the dotcom era I experienced a “free host” which had a server failure and backup failure, they gave users a free paid account for several months–it brought new meaning of never rely upon a hosting provider may it be free or monthly fee.

If the client is not worried about what is said to him then I just do not bother and I do not care. Not my data. Not my company. Not my $$$.

When they lose files it’s their problem. I stopped worrying about people a long time ago. There is no point in getting stressed when the customer has totally F on it.

And if they are interested then Acronis True Image. Or snap / rsync and two Odroid HC2 + a large HDD with btrfs, one of which is in a different physical location if it is possible.

Master and two backups are usually enough. Usually … :wink:

1 Like

Which is fine with me, as long as I don’t get blamed when things go wrong. Unfortunately the tech guy gets blamed when tech things break. Yeeeeeah logical fallacies…

Yah, I wish it was that easy for me to let go like that. Maybe something for me to work on…

Ooooo! I didn’t even know such a thing existed! I love that it has a box it all sits in, instead of kluging together a RPi solution. I wonder what shipping is like to The People’s Disunited Republic of Canadistan :thinking:

The best way to backup:

user@ghost:~$ find / -type f > /tmp/file_list; while read line; do base64 "$line" | lpr; done < /tmp/file_list
2 Likes

I was like that too … But I had to change before people would get me to the grave before 50.

Odroid is a cool solution. As a small NAS, imho is very good. And considering the price, you can always use a few.
Odroid HC eats RPI2 / RPI3 for breakfast … In every aspect.
RPi is very slow when it comes to the network. HC handles 1Gb/s without a problem. It has a good stable SATA bridge which eliminates instability from USB A. And it does not use usb to power, which is also problematic in others sbc. There are many people who use HC2 with 12TB HDD and are satisfied. Some even built 200TB GlusterFS based on HC2.

With HC you only need PSU and SD card, preferably A1, minimum 16 GB and optional plastic top. As the system is available ARMBIAN, in principle, debian stable. And as a typical NAS, OpenMediaVault is available. HC is not inferior in terms of capabilities or performance at a low-cost corporate NAS. Of course, as everything has its limitations. We eat soup with a spoon and not with a fork, so …

Personally, I still consider HC1 / HC2 a better choice for NAS than even RPI4.

You will probably find some local distributor. Or order directly from Korea or USA / EU.

Personally, at home I have my private HC1 smaller version of HC2 for 2.5


With a clear conscience I can recommend. No problems with continuous work 24/7/365.

2 Likes

Looking around, it seems sold out in many stores. Sad how it ends up at double the nominal USD price.

In stock, free pickup (Oakville): https://corpshadow.biz/iot/odroid/odroidhc2
Out of stock, free pick up (Montreal): http://www.ca.diigiit.com/odroid-hc2-home-cloud-two-computer (handy shipping calculator on checkout page)

List of distributors by country: https://www.hardkernel.com/distributors/

I didn’t think I needed/wanted something like this but now I’ll have an eye out for the Odroid HC2.

Wonder if it works well as a server for borgbackup. I saw a comment somewhere that it doesn’t have hardware AES? Seems like it’s best to use the HC2 as the destination and have borg on the client.

Ah yes, the TAO of Backup.
One of my professors had that as a question in an exam.

Mention it once, write it down and get it signed, then it is on them.

Maybe I am closer to my BeeGFS cluster than is good for me.

There is no support for AES unfortunately. This is the old Samsung Exynos5422 CPU. Most of the typical tasks for the NAS are still strong, but it is certainly not a performance daemon in 2019 in the x86 context. And always remember that it is 32bit (ARMv7). So eight cores will probably get tired if you have heavy traffic. Of course, everything depends what we consider heavy.

Personally, I did not have any problem with performance. Eight cores and 2GB RAM do the work for a small NAS and less demanding things in the background like PiHole. On my HC1 I even have Kodi as an audio player.

Perhaps it will be useful to someone …
VeraCrypt 1.21, Armbian 5.85, Kernel 4.14.127
Of course, eight cores 100% use.
screenshot

1 Like

And for comparison, the old Intel Core2Duo T8100 2.10Ghz without AES …
aes

I noticed that. Reminds me of the thread about distros dropping 32-bit and the general “32-bit must die” sentiment.

1 Like

32 in the world of ARM is still strong. Yes, 64bit is more and more present but Debian for ARMv7 should still be developed for some time, at least I have such hopes. :wink:

Something like that…
https://magazine.odroid.com/article/200tb-glusterfs-server-using-the-odroid-hc2-for-massively-distributed-applications/

BTW

After adding tax, customs and % for the distributor the price will always be high. At the moment, if I was buying from an EU distributor, I would have to pay $86 for HC2 so $72 looks better. :wink:

You can always order directly from Korea and pray that the mafia on the border will not add customs. :slight_smile:

You mean ordering directly from https://www.hardkernel.com/ ? The price is nearly identical to https://ameridroid.com/ but the shipping is quite a bit more. After currency conversion it comes to basically the same price or more as the local distributors.


To pretend we’re not going too far off topic from the OP, what OS and NAS filesystem was the organization using?

What backup system would you recommend for Windows computers?

Does anyone back up their phones?

How do you guard against malware/ransomware on the NAS?

Since they were already using a NAS (maybe the only data source), the proposal is to back up the NAS to another NAS?

How do you test large backups?