How does this [t]ricky bracket expression in grep work?

The square bracket expression is part of the bash shell (and other shells as well) grep's character class pattern matching.

The grep program by default understands POSIX basic regular expressions. With that you can define character classes. For example ps -ef | grep [ab9]irefox would find "airefox", "birefox", "9irefox" if those existed, but not "abirefox".

The command grep [a-zA-Z0-9]irefox would even find all processes that start with exactly one letter or number and end in "irefox".

So ps -ef | grep firefox searches for lines with firefox in it. Since the grep process itself has "firefox" in it, grep finds that as well. By adding a [], we are only searching for the character class "[f]" (which consists of only the letter "f" and is therefor equivalent to just an "f" without the brackets). The advantage of the brackets now is that the string "firefox" no longer appears in the grep command. Hence, grep itself won't show up in the grep result.

Because not very many people are familiar with square brackets as character class matching and regular expressions in general, the second result might look a little mysterious.

If you want to fix the second result, you can use them this way:

ps -ef | grep [f]irefox  > data
grep firefox data

(Reference)

The reason is that the string

grep firefox

matches the pattern firefox, but the string

grep [f]irefox

does not match the pattern [f]irefox (which is equivalent to the pattern firefox).

That's why the first grep matches its own process command line, while the second doesn't.