How can I remove a file or directory called "\"?

Use rm \\ (escape the backslash with another backslash). Note that this also works similarily, for directories named \ (using either rmdir, or rm with the -r flag).

Example:

>mkdir demo
>cd demo
>touch \\
>ls -l
total 0
-rw-------  1 hennes  users  0 Jul 29 20:25 \
>rm \\
>ls -l
total 0

A general tactic for manually deleting files with awkward characters in their names is

rm -i ./*

This will prompt you to choose whether or not to delete each file in the directory.

You can also unlink by referencing the inode of a file

linus ~/test $ touch \\
linus ~/test $ ls -li
total 0
15204561 -rw-r--r-- 1 pat sudo 0 Jul 29 23:03 \
linus ~/test $ find . -inum 15204561 -exec rm -v {} \;
removed `./\\'
linus ~/test $ ls -li
total 0
linus ~/test $