Apple - How to use Character Viewer with only keyboard and no mouse?

The Character Viewer has two states, as seen below.

  1. The complex state, that needs to be clicked first and doesn't seem to allow keyboard-only activation:

enter image description here

  1. The simple state, that gets automatically activated when invoked and is a joy to work with using only the keyboard. Tab key is used to switch between emoji categories:

enter image description here

You can switch between the states by clicking the friendly but almost invisible icon in the top-right corner (don't know what it's called, doesn't have any tooltip).

Friendly but invisible icon

To summarize, click the friendly icon, activate the Character Viewer, stay with the keyboard.