Hey all,
If you’ve seen my recent build log of my second Threadripper system — this time around, a beefy Workstation — I used a neat trick to get up and running rather quickly. All of mmm say 15mins?
When building my first Threadripper box, at the time I was just getting into Fedora 26 and I suppose, intentionally, that SSD took on the following form
-
Fresh Fedora 26 build
-
Decent partition scheme to start off; never got around to expanding the free-space in
/opt
.NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 238.5G 0 disk ├─sda1 8:1 0 448M 0 part /boot/efi ├─sda2 8:2 0 46.6G 0 part / ├─sda3 8:3 0 29.8G 0 part [SWAP] ├─sda4 8:4 0 14G 0 part /home └─sda5 8:5 0 21G 0 part /opt
-
Basic essential security tasks that I always perform along with replacing Fedora’s default firewall with
ufw
just cause I prefer it. -
Copied over all my production SSH keys // yes, I need to hold these in a LUKS encrypted partition, but that’s for a future enhancement.
-
Installed docker and setup my
rtmp
Twitch streamer (as my virtualised “server” stack doesn’t exist just yet). Yeah, I’m also running an internal Ghost blog for kicks.[mdesilva@ballinripper ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f45b795028fa ghost "docker-entrypoint..." 12 days ago Up 20 hours 0.0.0.0:1940->2368/tcp ghost 75d1b1da7e94 rtmp_master "nginx -g 'daemon ..." 3 weeks ago Up 20 hours 0.0.0.0:1935->1935/tcp twitch-rtmp
-
Setup
/opt
rather nicely - it has Nvidia drivers, some custom kernel modules etc. I already have some scripts in place for IOMMU spelunking, couple ISOs forvirsh
etc.[mdesilva@ballinripper ~]$ ll -sh /opt total 72K 4.0K drwxr-xr-x. 2 root root 4.0K Sep 16 15:26 drivers 4.0K drwxr-xr-x. 3 root root 4.0K Oct 27 13:16 ghost-docker 4.0K drwxr-xr-x. 3 root root 4.0K Sep 16 14:57 google 8.0K -rw-rw-r--. 1 mdesilva mdesilva 5.9K Sep 22 01:29 iommu_groups 4.0K drwxr-xr-x. 2 mdesilva mdesilva 4.0K Sep 19 12:52 isos 4.0K drwxrwxr--. 3 root wheel 4.0K Sep 19 14:08 libvirtd 16K drwx--x---. 2 root root 16K Sep 16 14:48 lost+found 4.0K -rwxrwxr-x. 1 mdesilva mdesilva 159 Sep 22 01:28 ls-iommu.sh 4.0K -rw-rw-r--. 1 mdesilva mdesilva 3.9K Sep 22 01:37 lspci_list 4.0K drwxrwxr-x. 4 mdesilva mdesilva 4.0K Sep 22 14:28 modules 4.0K drwxrwxr-x. 3 mdesilva mdesilva 4.0K Sep 20 09:30 projects 4.0K drwxr-xr-x. 3 root root 4.0K Sep 30 04:16 stability-testing 4.0K drwxrwxr-x. 4 mdesilva mdesilva 4.0K Oct 13 12:04 twitch 4.0K drwxrwxr-x. 3 mdesilva mdesilva 4.0K Oct 27 11:41 twitch-nginx-rtmp-docker
-
Woah, I forgot that I’d even setup a VM on the Gigabyte Threadripper box. On the
ballinripper
post-clone it has auto-magically booted up since I’d confirgured the Asus UEFI to enable virtualisation …and it has got itself an IP over DHCP and connecting to it works too![root@ballinripper ~]# virsh list --all Id Name State ---------------------------------------------------- 1 ubuntu-vm1 running
-
I had installed the Phoronix benchmarking test-suite. The Cryptography test is a particularly mid-range workload with
john-the-ripper
etc.phoronix-test-suite benchmark cryptography
-
Black-listed the Nouveau drive in
grub
withrd.driver.blacklist=nouveau
; it helps when installing the Nvidia closed-source drivers. -
Fedora only needed a quick update with
dnf update
As an initial starting point, this wasn’t too bad. So, I had the new(er) Threadripper workstation on the bench, passing POST… what does one do? break out dd
.
Update – However, since completing this backup, I found an even better alternative dcfldd
which as a “forensic” tool has some sweet hashing, error logging features. In a future version, I’ll most certainly upgrade my script to use this tool.
I wanted to backup the SSDs to image files, and so I hooked up a WD Black 4TB drive to my trusty StarTech Dual-Bay SSD/HDD Toaster (USB3.1 ftw)… and got scripting.
Mind you, this is a fresh ‘rehash’ for public consumption based on my initial crude attempt and isn’t 100% tested – but at least, it’ll serve as a nice starting point for anyone getting started.
- Didn’t bake in a menu-system or arg support, as this was quick-n-dirty.
- Specify settings via env vars at the top of the script.
- Make sure you pick the correct
input
source device, which is the backup source. - Intentionally left out
dd
'ssync
option and skip-errors option; I wanted to be notified of any read-errors. - Packed into a tarball, which admittedly, compresses already gzipped images. Oh well.
- Managed to bake 500GB of SSD ‘block-level’ data into a 79GB tarball.
- Checksums are manually generated. The aforementioned
dcfldd
does this ‘out of the box’ and can even include error logging, so I’d suggest using that!
Update – I’ve removed the old version;
See improved version in Post #5 below - HOWTO: Clone your SSDs & Boot Drives (for Archival ..and other things)
Once the backup is done, I wanted to verify the generated files. For this task I whipped up a simple Makefile
,
[root@ballinripper wdblack]# make verify
md5sum -c ballinripper_ssd_backups_08112017.chk
ballinripper_fedora26_backup_bs64K_08112017.img.gz: OK
ballinripper_fedora26_list_fdisk_08112017.info: OK
ballinripper_windows10_backup_bs64K_08112017.img.gz: OK
ballinripper_windows10_list_fdisk_08112017.info: OK
Both my SSDs are now backed up, and images are now stored on my FreeNAS pile o’ rust. Leave a like and thoughts - how have you tackled this in the past?
I’ve scripted automated backups and uploads to AWS S3 and may tailor this setup to do the same, such as 3-5 images held in rotation, per SSD. It would be neat to have a daily backup in the cloud should I ever need it, but that’s a later task. @SgtAwesomesauce @wendell @ryan @Raziel
Thanks!!