Calculate MD5 checksum of a directory

Solution 1:

Sure - md5sum directory/*

If you need something a little more flexible (say, for directory recursion or hash comparison), try md5deep.

apt-get install md5deep
md5deep -r directory

To compare a directory structure, you can give it a list of hashes to compare against:

md5deep -r -s /directory1 > dir1hashes
md5deep -r -X dir1hashes /directory2

This will output all of the files in directory2 that do not match to directory1.

This will not show files that have been removed from directory1 or files that have been added to directory2.

Solution 2:

If you'd like to see what's different (if anything) between two directories, rsync would be a good fit.

rsync --archive --dry-run --checksum --verbose /source/directory/ /destination/directory

This will list any files that are different.

---

Solution 3:

i think i answered this one before with this answer:

find . -xtype f -print0 | xargs -0 sha1sum | cut -b-40 | sort | sha1sum

gives: b1a5b654afee985d5daccd42d41e19b2877d66b1

the idea is you hash all the files cut out the hashes one per line, sort them and hash that yielding a single hash. this doesn't depend on the names of the files.

---

Solution 4:

The cfv application is quite useful, not only it can check and create MD5 checksums, it can also do CRC32, sha1, torrent, par, par2.

to create a CRC32 checksum file for all files in current directory:

cfv -C

to create a MD5 checksum file for all files in current directory:

cfv -C -t md5 -f "current directory.md5sums"

To create a separate checksum file for each sub directory:

cfv -C -r

To create a "super" checksum file containing files in all sub directories:

cfv -C -rr

---

Solution 5:

I used hashdeep, as explained in this askubuntu answer: Check the correctness of copied files:

To calculate the checksums:

 $ cd <directory1>
 $ hashdeep -rlc md5 . > ~/hashOutput.txt

To verify and list the differences:

 $ cd <directory2>
 $ hashdeep -ravvl -k ~/hashOutput.txt .
 hashdeep: Audit passed
    Input files examined: 0
   Known files expecting: 0
           Files matched: 13770
 Files partially matched: 0
             Files moved: 0
         New files found: 0
   Known files not found: 0

This has an advantage over md5deep in that it will show renamed (moved), added, and removed files, as well as avoiding the problem with 0 length files pointed out at the bottom of http://www.meridiandiscovery.com/how-to/validating-copy-results-using-md5deep.