Btrfs partition rearrange and resize gone wrong, not able to find filesystem

Software & Operating Systems Linux
,
1

i tried resizing and rearranging my partitions with kde partition manager on a live USB, the idea being to take my boot drive and prepare it for a dd transfer to a faster drive i had. unfortunately, the operation failed after some 4 hours on the final step, checking the partitions. the end result is now i have no idea whats actually where on the disk, and i could really do to recover some of the configuration files, notes on things, and scripts i had stored in my home directory.

testdisk doesnt seem to find much of anything, sudo btrfs filesystem show /dev/sde yields nothing, sudo btrfs check /dev/sde finds no valid filesystems on it, all permutations of sudo btrfs inspect-internal dump-super with respect to existing partitions and superblock numbers yields either a blank result or ERROR: bad magic on superblock. im not even sure that the existing partitions line up with where the data would be, given i both re-arranged and shrunk various partitions.

beyond this i have no idea how best to proceed with data recovery. id wager theres a chance my configs and scripts exist on the disk to be found, but i dont know what tools or procedures i aught to be using on linux with respect to btrfs. anyone got any tips or things i should look at or into?

2

This is the “restore from backup” situation.

If you care about any remaining data on it, maybe get another drive and start reading btrfs code to understand chunk structures and to try and figure out where things went wrong. Good news is that because of checksumming it should be possible to recover. Bad news is that it’s complicated.

1 Like
3

well that sounds like a fun way to spend a week kek. if i go about trying to find individual files, is there any tool for that on linux? iirc theres plenty of windows tools with that sort of functionality for raw disks.

4

PhotoRec is a recovery tool I have heard of before, but I do not know if it has specific support for btrfs.

However; first, it sounds like you have been pointing the btrfs tools at the bare drive (/dev/sde) rather than a partition. Are any partitions appearing in either in /dev/ as sde_ or using a utility like gdisk (CLI), fdisk (CLI), GNOME Palimpsest/Disks/Disk Utility (GUI), Gparted/Qtparted (GUI), or KDE Partition Manager (GUI).

Maybe you should try recovering the partition table first? Are you using GPT or MBR as your partition table?

Edit: ERROR: bad magic on superblock — Magic numbers in file formats act as a sort of identifier, so if btrfs is not seeing the proper “magic” it probably means that it is not looking in the right place for the superblock.

5

the partitions all have the layout i wanted after the operation was complete, but some are unknown format according to kde partition manager. i have tried pointing said tools both at the raw disk device, as well as the partitions themselves, all to no avail.

im using GPT, but i know of no way to recover the partition table without a backup. i should have taken one before i started but i did not, unfortunately.

6

I would then try running hexdump on at least the first 512 bytes (-n 512) of the mystery partitions, in the canonical format (-C) to get a sense of what is going on there:

sudo hexdump -C -n 512 /dev/sde_
7

good call. the first mystery partition is a valid NTFS one so apparently my windows partition was moved without issues, but the one im mainly interested in in the BTRFS linux one, which produces garbled output upon inspection of the partition its supposed to be in with hexdump.

8

It looks like the normal state of BTRFS is to store the first superblock at 0x10000 (64 KiB), and the second at 0x4000000 (64 MiB). On my test BTRFS partition, everything below 0x10000 is all zeros, so when I run:

sudo hexdump -C -n 0x10100 /dev/sd…

I see roughly this output (being somewhat paranoid, I have stripped out any possible identifying sections):

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00010000              00 00 00 00  00 00 00 00 00 00 00 00  |    ............|  // only 4-bytes of 32 available
00010010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|  //  used by crc32c checksum
00010020                                                    |                |  // 16-byte Filesystem UUID
00010030  00 00 01 00 00 00 00 00                           |........        |  // 8-byte Address + 8-byte Flags 
00010040  5f 42 48 52 66 53 5f 4d                           |_BHRfS_M........|  // 8-byte Magic + 8-byte Gen.
00010050                                                    |                |  // 8-byte Root + 8-byte Chunk
00010060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|  // 8-byte Log + 8-byte TransID
00010070                                                    |                |  // 8-byte Total + 8-byte Used
00010080  06 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|  // 8-byte Objectid + 8-byte Devices
00010090              00 40 00 00                           |    .@..        |  // Sector + Node + Leaf + Stripe  
000100a0                                                    |                |  // Array + Chunkgen + Compat-
000100b0                                                    |                |  //  + CompatRO + Incompat-
000100c0              00 00                                 |    ..          |  //  + Csum_type + Levels + DEV_ITEM-
000100d0                                                    |                |  //  DEV_ITEM continued
000100e0                                                    |                |  //  DEV_ITEM continued
000100f0                                                    |                |  //  DEV_ITEM continued
00010100

The magic number to look for is that text _BHRfS_M in ASCII, sitting at offset 0x40 within the superblock. The full documentation for the superblock format is on kernel.org:

https://btrfs.wiki.kernel.org/index.php/On-disk_Format

That is strange; as mentioned above the superblock is actually much further within the partition; if you ran the command on a proper BTRFS partition, I think you should have just seen null bytes. For me to see anything, I had to hexdump all the way out to address 0x10100 (512 B more than 64 KiB).

Absurd Hack to Hunt for a Superblock

To try to find the superblock, my crazy solution would be to try:

sudo strings -a -n 8 -t x /path/to/partition | grep _BHRfS_M

and watch to see where/if the superblock magic value shows up.

This is running strings in scan-the-whole-file mode (-a), looking for strings 8 bytes long (-n 8), giving us the location in the file in hexadecimal (-t x); then piping to grep, which searches for the particular string we are looking for (_BHRfS_M).

Next steps

Once we know where the a superblock is, you can hexdump just the surrounding area with the skip (-s __) and length (-n __) arguments.

If we hexdump a superblock, maybe we can craft a restore command to extract some data?
https://btrfs.wiki.kernel.org/index.php/Restore

3 Likes
9 
2f01f92895  5f 42 48 52 66 53 5f 4d  09 42 54 52 46 53 20 46  |_BHRfS_M.BTRFS F|
2f01f928a5  69 6c 65 73 79 73 74 65  6d 0a 3e 30 78 31 30 31  |ilesystem.>0x101|

Huzzah, found it. or one of them at least, assuming /dev/sda contains the first superblock i may be in luck. would not have thought to try searching the entire drive like that tbh, for some strange reason my brain read that as a “load the entire disk into RAM before doing anything” operation rather than a sensible streaming operation at first.

10

Hmm, that does not look correct; I would keep searching.

The area surrounding the Magic value should not contain any text:

Offset 8 Bytes 8 Bytes
30 bytenr (where this superblock is)
physical address 		flags
bitmask
40 "_BHRfS_M"
string 		generation
integer
50 root tree root
logical address 		chunk tree root
logical address

My guess is that you happened to find the executable for a program that needs to parse BTRFS. The executable code for a partitioning program will need to have a copy of the magic value to know what to look for after all.

Ideally, we want to find two superblocks that are 64 MiB apart; then we will know where BTRFS thinks that the partition starts. My guess is that somehow your partition editor changed the partition table without actually moving data.

11

I am getting a bit confused, so thought I would draw up a quick summary:

Cause

KDE Partition Manager running move + shrink partition operations (in preparation for a dd drive transfer that never occurred, due to present issues)

KDE Partition Manager Failed after 4 hours during partition check step.

Repair attempts

  • testdiskfinds little
  • btrfs filesystem show /dev/sde yields nothing
  • btrfs check /dev/sde finds no valid filesystems
  • btrfs inspect-internal dump-super “all permutations of”
  • hexdump the start of mystery partitions; one is NTFS, other or others still unknown
  • strings+grep search for BTRFS magic value (_BHRfS_M) finds a red herring at 0x2f01f92895 - a random string on-disk of “_BHRfS_M.BTRFS Filesystem.>0x101”, but not a superblock

Is this correct?

What options did you try with dump-super? For example, did you try the -a option?

Did the strings+grep search turn up any other instances of the magic value?

12

yep, sudo btrfs inspect-internal dump-super -a -s N /dev/sde_ where N ranges 0-2 and _ is numbers 3 and 6, because those align to where partitions were as far as i can remember, and where they should now be.

strings + grep yielded a ton of results for that search, havent been through them all yet tbh. might fiddle with things a bit and re-run the search to see if i can narrow it down without having to hedump each of them.

13

Hmm, in most cases the surrounding bytes will not be valid ASCII; maybe change the grep command to instead be:

grep -E '^\s*[0-9a-f]+ _BHRfS_M$'

to limit to the false positives?
That regex is: from the start ^, any number of whitespace characters \s*, one or more characters of hex [0-9a-f]+, a space  , magic string _BHRfS_M, end of line $.

I am having trouble imagining this matching any false positives from the strings output, so I assume it will work. False negatives are only likely if …

False Negatives are definitely a possibility; that generation integer that I mentioned comes directly after the magic value? It is in little-endian, which means that it is entirely possible for the first byte to be a valid ASCII character; if the above grep does not work, I would next try:

grep -E '^\s*[0-9a-f]+ _BHRfS_M'

which removes that trailing $ to allow for valid ASCII characters in the first bytes of that generation integer.

My thinking here is to attempt:

  1. IN PROGRESS - find superblocks (non-destructive)
  2. if that succeeds, then: btrfs restore (non-destructive)
  3. if that fails, only then try: btrfs rescue chunk-recover (possibly destructive)