How to search text throughout entire file system?

I normally use this style of command to run grep over a number of files:

find / -xdev -type f -print0 | xargs -0 grep -H "800x600"

What this actually does is make a list of every file on the system, and then for each file, execute grep with the given arguments and the name of each file.

The -xdev argument tells find that it must ignore other filesystems - this is good for avoiding special filesystems such as /proc. However it will also ignore normal filesystems too - so if, for example, your /home folder is on a different partition, it won't be searched - you would need to say find / /home -xdev ....

-type f means search for files only, so directories, devices and other special files are ignored (it will still recurse into directories and execute grep on the files within - it just won't execute grep on the directory itself, which wouldn't work anyway). And the -H option to grep tells it to always print the filename in its output.

find accepts all sorts of options to filter the list of files. For example, -name '*.txt' processes only files ending in .txt. -size -2M means files that are smaller than 2 megabytes. -mtime -5 means files modified in the last five days. Join these together with -a for and and -o for or, and use '(' parentheses ')' to group expressions (in quotes to prevent the shell from interpreting them). So for example:

find / -xdev '(' -type f -a -name '*.txt' -a -size -2M -a -mtime -5 ')' -print0 | xargs -0 grep -H "800x600"

Take a look at man find to see the full list of possible filters.


Normally you wouldn't want to actually search EVERYTHING on the system. Linux uses file nodes for everything, so some "files" are not things you would want to search. For example /dev/sda is the physical block device for your first hard drive. You probably want to search the mounted file systems not the raw disk device. Also there is /dev/random which spits out random data every time you read it. Searching that doesn't make a lot of sense. The /proc file system is also problematic in your case.

I would recomend one of two things.

  1. Don't search at root, only search the places that might be useful. Search /home or /usr or /etc separatly. The info you are looking for is likely of a specific type, so it's likely to be in a specific folder anyway. Configuration settings should be in /etc. Your personal data files should be in /home. Limiting search to a major area like this will greatly reduce your problems with recursive greps.

  2. Exclude problematic areas using --exclude-dir and a set of things you know you don't needlike this:
    grep -r --exclude-dir /proc --exclude-dir /dev --exclude-dir /tmp --exclude-dir /lost+found

Lastly, it's not uncommon to run across a few 'permission-denied' errors when doing a big recursive grep. In the normal course of use there are files your user may not be able to read. As long as these are just a few odd files and not things like the raw device for your hard drives or the entire proc file system, it's ok to just ignore the errors. In fact you can do this on the command line by sending all the errors into never never land:

grep -r search_string /path 2> /dev/null

For simplicity, I would suggest ack-grep. Link shows many cases when ack-grep is a better option.

To use is, after install:

ack-grep pattern /

Tags:

Grep

Recursive