Rsync - move files with preserving hardlinks outside transfer set

The problem

I am using unRAID on my home server. I use Manual split level on user shares to keep files together, and not scattered all over the different drives. This means, that quite often I am manually moving folders around from one disk to another. I also use a lot of hardlinks for torrents. Actually, all my seeding torrents are hardlinks.

Here’s a simplified example of the folder structure. There is actually only 2 movie files, both having 2 hardlinks.

/mnt/disk1/media/movies/movie1/movie1.mkv
/mnt/disk1/media/movies/movie2/movie2.mkv

/mnt/disk1/media/torrents/movie1.mkv
/mnt/disk1/media/torrents/movie2.mkv

Now let’s say I want to move movie1 directory to disk2, and preserve all hardlinks, if there’s any.

from:  /mnt/disk1/media/movies/movie1
to:    /mnt/disk2/media/movies/movie1

Here’s what I do now… manually

I use a bash script to check if the directory has any files with hardlinks outside that directory, because:

Note that rsync can only detect hard links between files that are inside the transfer set. If rsync updates a file that has extra hard-link connections to files outside the transfer, that linkage will be broken.

The script uses find with -samefile to find all hardlinks for all the files, that have any.

For the above example, I would have found, that movie1.mkv has a hardlink inside torrents directory.

So, I manually move everything to the same temporary move folder.

/mnt/disk1/media/move/movies/movie1/movie1.mkv
/mnt/disk1/media/move/torrents/movie1.mkv

Then, I use rsync to move the files (preserving the hardlinks).

cd /mnt/disk1/
rsync -avPRXH --remove-source-files media/move /mnt/disk2/

Afterwards I delete the leftover empty directories.

find /mnt/disk1/media -type d -empty -delete

And finally I manually move the files from the move folder back to their original locations.

/mnt/disk2/media/movies/movie1/movie1.mkv
/mnt/disk1/media/movies/movie2/movie2.mkv

/mnt/disk2/media/torrents/movie1.mkv
/mnt/disk1/media/torrents/movie2.mkv

Can rsync and/or/with some other linux tool do this automatically?

As you can imagine, doing this manually is really tedious, especially when moving large directories, with lots of hardlinks.

I could write another script to automate this, but writing bash scripts is real pain (imho, the syntax is so bad). So, atm I am writing a small cli app with node.js to automate this process.

But because I am a total Linux and command line newb :worried: and I know that actually I do not know anything… I decided to ask maybe there is a good script/tool to do that already? I mean, this feels like a common enough problem, that probably got solved countless times ages ago…

Thanks in advance for any help.

Did you ever find a neat solution for this?

I’m trying to do exactly the same, migrate TV to one disk, and movies to another

Wondering if using include and exclude while setting up working dir to /mnt/disk1/media will solve the problem