Mtree reports files missing which aren't

Operating Systems & Open Source
1

I was looking for a reasonably fast way to compare directory hierarchies. mtree seemed like a good candidate. However, when I started experimenting with it, I quickly got confused. I started with two datasets on two different TrueNAS systems; ran mtree on one, and then used mtree to check the other. It reports: file-1: missing; however, file-1 isn’t missing. I can list it, compare it, etc. In fact, “diff -r” shows everything is identical. What gives? Here’s a snipit of a trial I did. Hopefully someone can enlighten me. By the way, mtree on Debian and FreeBSD give (modulo some formatting) the same results. Oh yeah, the snipit below is checking a run against itself.

[user@fbsdcoff1 ~]$ sudo mtree -c -R all -p /net/coffer/mnt/tank/Test_Data/ownCloud/clientsync/ > two
[user@fbsdcoff1 ~]$ sudo mtree -f two -p /net/coffer/mnt/tank/Test_Data/ownCloud/clientsync/
./#scratch1.txt# missing
./#scratch_tax.txt# missing
./Copy_scratch_drive_to_new_drive_via_rsync.txt missing
./scratch missing
./scratch.txt missing
./scratch1.txt missing
./scratch2.txt missing
./scratch3.txt missing
[user@fbsdcoff1 ~]$

2

I was looking for a reasonably fast way to compare directory hierarchies.

Maybe try something like this, replace diff with vimdiff for larger outputs.

$ diff -u <(cd a; find . | sort) <(cd b; find . | sort)
--- /dev/fd/63	2024-03-10 17:52:45.600604314 +1030
+++ /dev/fd/62	2024-03-10 17:52:45.600604314 +1030
@@ -1,3 +1,3 @@
 .
-./1
+./2
 ./3
3

I should explain my goals using mtree. I have multiple NAS systems and want to check that the datasets across them are the same. It’s too time consuming to do multiple recursive diffs on a regular basis. So, I’ve written simple scripts to help assure me things are okay. Every day a script runs which simply compares file listings between the NAS systems. It’s not much, but if files are missing, I’ll know. This was why I wanted to use mtree.

On a longer interval, I run cfv across the datasets and see if the file checksums match. Here, I could run mtree with one of the cksum options – probably just a 32 bit CRC.

This “project” made it on to my radar lately as I had a file silently corrupted in some fashion between two of the systems. cfv alerted me to this problem. I imagine this was the result of corruption during the initial copy to the new NAS, but don’t know. I used a zfs send/recv pipeline to make the initial copies.

FYI, all of the systems are running TrueNAS. The main NAS is an old Dell server running CORE, both the backup systems are commercial iXsystems TrueNAS boxes; one running Scale, the other running CORE. The plan is to retire the old Dell and run with the two iXsystems.

I’ve resorted to digging through the mtree source trying to figure out why it’s behaving as it is. However, I don’t know that I’ll have time to continue with that.

4

Well. It appears that mtree doesn’t properly handle horribly named files. The directory I was using to test has a file named

*scratch*

and some other files with the string “scratch” as part of the name. So the machinery breaks when given a file containing the asterisk indicating a wildcard.

That file goes back to 2017. Deity only knows what I was doing at the time to create such a monstrosity. So, it’s a bug in the code, BUT an understandable one.