(I debated on reviving another older thread on QNAP data, decided against it because that thread seems to have died in 2018.)
Friday night; found that my TS-253B was no longer responding on the network. We’ve had some power outages recently and I assumed I had intentionally powered it off. Went to the cabinet, saw it was on. I force-powered it off and as soon as it went off the status light came back on red and started flashing.
This wasn’t a problem I felt like dealing with on a Friday. I’ll just go get the file I needed from my backups. Oh good I never finished getting those set up for some reason.
(I’m going to shoutout @jrcichra here at the beginning for dealing with my incessant questions through this)
Step 1 on the road to recovery was just to get the drives copied so we can try to recovery from copies without risking the drives. I had some issues with my SATA docking station playing nicely on my server, so I just pushed the data from another machine on the network. Annoying, but doable. I’m not sure why exactly, but once the read side dd had completed, it didn’t close, so the write side also hung open. Once they stopped transferred, I just terminated them. It didn’t’ cause me any problems, I just thought I’d mention it.
Server Side:
nc -l 21001 | lz4 --decompress dd of=/z4x14/qnap_recovery/WD40EFRX-68WT0N0.img bs=4k status=progress
Workstation Side:
dd if=/dev/sde status=progress bs=4k | lz4 --compress | nc 10.0.0.252 21001
Step 2 was to validate the drives and images had the same data. md5sum to the rescue. I am not hugely confident in using the the md5sum command to automatically compare the hashes, but I know how to write them to a file. This also gave me the opportunity to see how far along the drive those were.
Server Side:
pv /z4x14/qnap_recovery/WD40EFRX-68WT0N0.img | md5sum > /z4x14/qnap_recovery/WD40EFRX-68WT0N0.img.md5sum
Workstation Side:
pv /dev/sde | md5sum > WD40EFRX-68WT0N0.disk.md5sum
Output from those files. I know there’s better ways to do it.
WD40EFRX-68N32N0.disk.md5sum:fda5493b4dbbb784e459c75ff499a7ce -
WD40EFRX-68N32N0.img.md5sum:fda5493b4dbbb784e459c75ff499a7ce -
WD40EFRX-68WT0N0.disk.md5sum:dc76f7c11fb29a7a2a60bf1445efd9ab -
WD40EFRX-68WT0N0.img.md5sum:dc76f7c11fb29a7a2a60bf1445efd9ab -
Step 3 was to spin up a VM and try an seemingly endless list of things. It seems like it should have been just mdadm --assemble and then mount the ext4 filesystem that is presented. mdadm does present a drive; and you can see it. but then there’s a drbd volume on there which even once we got a working drbd config wouldn’t read correctly. Tried using binwalk to find the filesystems and then mount -o offset=nnn but that gave an error about the filesystem needing cleaned.
One of the posts in This QNAP Forum Post lead me to basically believing this was never going to be possible.
…given that QNAP maintains their own fork of the Linux Kernel…
That got me back around to thinking that the nas is just an computer. It runs some software, surely we can get that virtualized. It was getting kind of late, but I pulled the TS-253B firmware from a QNAP post detailing how to recover from a failed firmware update. Messed with it for a couple hours and went to bed.
Step 4 Monday morning I remembered that QNAP offers their software as a “run on your own hardware” appliance. $5/month for a single core license. QuTScloud for private cloud if you. (I tried to throw in a link, it seems unhappy)
I figured that was well worth the price of admission and figured it was worth a shot.
Created a new VM without attaching the drives, did a basic install. (It did require me to attach a drive with at least 200GB of free space.) Once I was able to get logged into the normal web console and saw it was pretty much exactly what the physical nas had, I powered the VM off, attached the drives and was able to recover the storage pool in the VM.
From that point, It was just a matter of downloading the data.
Check on your backups before you need them.
I hope this helps somebody in the future, I was really stressed out about this.