How do I find out which processes are preventing unmounting of a device?

Use lsof | grep /media/whatever to find out what is using the mount.

Also, consider umount -l (lazy umount) to prevent new processes from using the drive while you clean up.

Most of the time, the best command to use is lsof (“list open files”).

lsof +f -- /media/usb0

where /media/usb0 is the mount point of the USB drive or other filesystem to unmount. +f -- tells lsof to treat the subsequent argument as a mount point; it usually, but not always, manages on its own, so that lsof /media/usb0 also works. This finds open files (even unlinked ones), memory mapped files, current directories, and some more obscure uses. You'll need to run the command as root to get information on other users' processes (and I think there are unices where lsof has to be run as root).

There are uses that lsof will not find; these are uncommon on removable media. They include:

• mount points: you can't unmount /foo if /foo/bar is a mount point.
• mount devices: you can't unmount /foo if /foo/bar is a mounted block device or loop-mounted regular file, or if it is the source of a Linux bind mount.
• NFS export: lsof won't detect that a tree is exported by a kernel NFS server.

Another command that can serve in a pinch is fuser, which only lists PIDs of processes with open files on the device:

fuser -m /media/usb0

Open files

Processes with open files are the usual culprits. Display them:

lsof +f -- <mountpoint or device>

There is an advantage to using /dev/<device> rather than /mountpoint: a mountpoint will disappear after an umount -l, or it may be hidden by an overlaid mount.

fuser can also be used, but to my mind lsof has a more useful output. However fuser is useful when it comes to killing the processes causing your dramas so you can get on with your life.

List files on <mountpoint> (see caveat above):

fuser -vmM <mountpoint>

Interactively kill only processes with files open for writing:

fuser -vmMkiw <mountpoint>

After remounting read-only (mount -o remount,ro <mountpoint>), it is safe(r) to kill all remaining processes:

fuser -vmMk <mountpoint>

Mountpoints

The culprit can be the kernel itself. Another filesystem mounted on the filesystem you are trying to umount will cause grief. Check with:

mount | grep <mountpoint>/

For loopback mounts (thanks Stephen Kitt), also check the output of:

losetup -la

Anonymous inodes (Linux)

Anonymous inodes can be created by:

• Temporary files (open with O_TMPFILE)
• inotify watches
• [eventfd]
• [eventpoll]
• [timerfd]

These are the most elusive type of pokemon, and appear in lsof's TYPE column as a_inode (which is undocumented in the lsof man page).

They won't appear in lsof +f -- /dev/<device>, so you'll need to:

lsof | grep a_inode

For killing processes holding anonymous inodes, see: List current inotify watches (pathname, PID).

inotify watches (Linux)

This comment explains why inotify shouldn't prevent an unmount, but this note describes the situations in which it will:

an unmount can hang in the vx_softcnt_flush() call. The hang occurs because inotify watches increment the i_count variable and cause the v_os_hold value to remain elevated until the inotify watcher releases the hold.