Why is rm slow on an external storage drive (USB-connected, type fuseblk) with 50Gb of files?

Ultimately, no matter what you do, rm has to run unlink on every single file that you want to remove (even if you call rm -r on the parent directory). If there are a lot of files to remove, this can take a long time.

There are two particularly time consuming processes when you run rm -r:

  1. readdir, followed by,
  2. a number of calls to unlink.

Finding all the files, and then going through every single file to remove it, can take a really, really long time.

If you find this "unusable" because it renders the directory unusable for some time, consider moving the parent directory before removing it. This will free up that name for the program to use again, without the time being too much of an inconvenience.

Assuming that the file system really is NTFS (it's unclear from your question), NTFS is generally quite slow at deleting large swathes of files. You might consider using a more suitable filesystem for your purposes (the more recent ext filesystems have pretty good delete performance, if you don't have any other particular needs). FUSE itself is also not particularly fast, in general. You might consider seeing if you can do this in some way that does not use FUSE.


Why is rm so slow? I have no idea. But I do know a faster way:

mkdir blank
rsync -a --delete blank/ test/

Update: This answer on Serverfault has some explanations. It looks like rsync is deleting the files in a particular order that causes the filesystem tree to remain balanced, and not ever need rebalancing. rm will just delete the files and cause a lot of rebalancing as they are removed. There is some information about rebalancing here.