Is there some kind of icon browser?

Instead of opening the folders one by one, you can also use the Search feature of Nautilus. Navigate to /usr/share/icons, and press the Search icon on the right of the toolbar.

Search for . (all images have an extension with a dot before it) and press Enter. On a default installation, this yields about 17.5k images. That's not much of an "overview", but it includes all files in /usr/share/icons.

If you wish to avoid searching each time, you could make use of symbolic links to the images: all images are accessible from one big folder.

  1. Open a terminal
  2. To check the number of files that can be created after creating the links:

    expr $(df /home -i | tail -1 | cut -d'%' -f1 | rev | awk '{ print $2 }' | rev) - $(find /usr/share/icons -type f | wc -l)
    

    You should not continue if the number is lower than 1000 and a negative number will cause the operation to fail after some time.

  3. Make an folder named icons-all by running: mkdir icons-all
  4. Go into that folder: cd icons-all
  5. Run nano /tmp/make-icons-link
  6. Paste:

    #!/bin/bash
    if [[ $1 == *.* ]]; then
            ext=".${1##*.}"
    else
            ext=
    fi
    name="$(basename "$1" "$ext")"
    extra=
    while [ -e "$name$extra$ext" ]; do
        ((extra++))
    done
    ln -s "$1" "$name$extra$ext"
    
  7. Press Ctrl + X, followed by Y and Enter
  8. Now generate the links, this may take a while:

    find /usr/share/icons/ -type f -exec bash /tmp/make-icons-link {} \;
    

    After the command has completed, no output is shown.

  9. Close the terminal by running exit
  10. The images are now visible in ~/icons-all. Loading this directory may take a while