Return to Level1Techs.com

Help recover my OS and data?

Hello. I did a very, very, very silly thing.

I ran fio on the block device on my Arch Linux OS drive, a 500GB 970 Evo.
I did not understand my colossal ef-up at the time and I rebooted.
I am now on an Ubuntu live USB disk having completed a full dd of the drive to my NAS.

The fio was programmed to run 4GB of random 4K writes on the drive. Needless to say it wreaked quite the havoc.

There are no partitions visible on the drive. I set my OS up as per the Arch Wiki.

I think I did /boot, /root, /home, /swap in this particular order. I don’t really remember the sizes of the partitions.

Anyone interested in helping me with some tech support?

Anything useful out of testdisk or photorec?

1 Like

+1 for photorec. Was able to completely recover a USB drive with that thing. Great piece of software.

2 Likes

testdisk /dev/nvme0n1 Analyze:

1 P EFI System 2048 1050623 1048576 [EFI System]
2 P Linux Swap 1050624 2099199 1048576 [Linux swap]
3 P Unknown 2099200 631244799 629145600 [Linux filesystem]
4 P Unknown 631244800 976773134 345528335 [Linux filesystem]

After hitting backup:

P MS Data 2048 1050623 1048576 [NO NAME]
P MS Data 2099200 631244799 629145600
P MS Data 631244800 976773127 345528328

What should I do next?

Edit: Got the fstab!:

cat fstab

Static information about the filesystems.

See fstab(5) for details.

/dev/nvme0n1p4

UUID=8bd88fb3-7e61-497d-bad7-66eb97168059 / ext4 rw,relatime 0 1

/dev/nvme0n1p1

UUID=FA78-97CC /efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 2

/dev/nvme0n1p2

UUID=137274c5-e86e-4b54-9c47-f08911870afa none swap defaults,pri=-2 0 0

/dev/nvme0n1p3

UUID=7c733278-7575-4c85-a6bf-e4c1c6eea95a /home ext4 nosuid,nodev,noatime 1 2

/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0

Please advise on how to proceed

Get another SSD to try and restore /home to?

I’m far from an expert here so I’m not sure if we can fix it but we should be able to get your home files out.

Completing testdisk, I get my partitions back.

The home partition does not want to mount. Running fsck I get a ton of errors. When it is completed, the home directory is empty and everything is in folders in the lost+found directory.

Some of the data will have likely been overwritten. The partition wont be good, but the data might. Thats where photorec comes in. Use it on your image to see what you can get back.

Well, I never got to recover the “root” of the home partition intact.

After an fsck, everything got shoved into the lost+found folder so I used this technique: http://karuppuswamy.com/wordpress/2010/06/09/how-to-recover-files-from-lostfound-after-fsck-in-linux-how-i-did-it-in-ubuntu/
to at least identify my Downloads folder, where most of my stuff was placed and got most of it back.

I also got some of the “root” files on the home partition back. I managed to get back (most) of the .config directory so I did not have to reconfigure everything as to get my setup back.

I ran Photorec and it has found lots of files but the naming is wrong so I have to manually go through it and find any files I am still missing or that have been corrupted and I don’t know about it yet. Meld is a good way to compare files and directories that might be of help if you happen to suffer from something like this.

The /root partition was absolutely intact for some reason. Fio probably thought better of overwriting it (or I just got lucky there since it was writing to the block device, not the partitions). So, it was just a matter of downloading an Arch bootable iso, setting up the EFI and swap partition again, correcting the fstab and generating the initramfs and kernel on the EFI partition and I mostly got my system back. I did reinstall all packages just in case something did get corrupted in the /root partition but everything works as it did previously. I now just have to sort out the files that I got back from the fsck and the files that Photorec managed to recover.

A backup would have saved me tons of time and frustration but I neglected it since I have been in the verge of turning my Arch install from a regular GPT partition install to moving /home and /root on ZFS so backing everything up to my ZFS NAS is easier, faster and less complicated. So I neglected backing up my files because I was in the process of making it easier for myself to back up my files :man_facepalming: :man_facepalming:

Moral of the story: always have a backup

1 Like