How can I delete a file named ">"?

Any of these should work:

sudo rm \>
sudo rm '>'
sudo rm ">"
sudo find . -name '>' -delete
sudo find . -name '>' -exec rm {} +

Note that the last two commands, those using find, will find all files or directories named > in the current folder and all its subfolders. To avoid that, use GNU find:

sudo find . -maxdepth 1 -name '>' -delete
sudo find . -maxdepth 1 -name '>' -exec rm {} +

You can also use Python to remove it:

python -c 'import os;os.remove(">")'

With POSIX find:

find . ! -name . -prune -type f -name '>' -exec rm -f {} +