Windows 10 + 11 Duplicate File Finder Question

Operating Systems & Open Source Windows
1

So currently I’m using a Freeware app named AUSLOGICS Duplicate File Finder for both Windows 10 and 11 Pro.

It does a pretty good job but I found one thing it can’t seem to do, that is find duplicates NOT based on the filename but rather a HASH of the file itself.

So for instance IF you have a file named foobar and foobar1 with the same file length, maybe they are the same file but without actually viewing/reading each file you can not know for sure.

In my case it’s basically all video or MP4 files in my media library that is spread across multiple nearline storage devices.

Is there a CLI or some other Windows app that can actually do this ?

Basically select two or more of the drives or folders located on multiple drives and then search for dupes based on the HASH of each file and then produce the list accordingly.

Any thoughts on this one ?

2

dupeguru!

3

Thanks, checking and testing it out now :slight_smile:

4

@FunnyPossum, thanks again for the tip, this app seems to allow me to find the dupes when the filenames are not the same.

Helping me clean up the library and frees up some disk space too :wink:

5

You will find that a lot with duplicate downloads of the same file, any os will add a number to it.
Providing the hash is the same you can just delete them.
There are a lot of good duplicate file finders out there. Its just a matter of users actually putting forth the effort to use them.

Because im a linux user i tend to voice my opinion on windows, But i will not tell people not to use windows.
Im also old school enough to know the importance of proper system maintainance and will gladly discuss methods in doing so.

6

I have used Duplicate Files Deleter tools to find and delete duplicates, it was worked well for me.

7

Seems like an easy job for PowerShell if you have it installed on your system.

<#
This script calculates the hash of a file and compares it to the hash of all files in a directory.
#>

#### GET THE HASH OF A FILE YOU WANT TO COMPARE
# Specify the path to the file
$filePath = "/Documents/ISO/filename.iso"
# Calculate the hash
$fileHashBase = Get-FileHash -Path $filePath -Algorithm SHA256
# Print the hash
$hash = $fileHashBase.hash


#### GET THE HASH OF ALL FILES IN A DIRECTORY YOU WANT TO SEARCH
# Specify the path to the directory
$dirPath = "/Documents/ISO"
# Get all files in the directory
$files = Get-ChildItem -Path $dirPath -File
# Calculate and print the hash for each file
foreach ($file in $files) {
    $fileHash = Get-FileHash -Path $file.FullName -Algorithm SHA256
    if ($fileHash.hash -eq $hash) {
        Write-Output "Duplicate file found: $($file.Name), Hash: $($fileHash.Hash)"
    }
}