Display Non-Zero Return Status in PS1

Make sure that the promptvars option is on (it is by default). Then put whatever code you like in PROMPT_COMMAND to define a variable containing exactly what you want in the prompt.

PROMPT_COMMAND='prompt_status="$? "; if [[ $prompt_status == "0 " ]]; then prompt_status=; fi'
PS1='$prompt_status\h \w \$ '

In zsh you could use its conditional construct in PS1 (bash has no equivalent).

PS1='%(?,,%? )%m %~ %# '

The best I can have is obtained by

PS1='${?/#0/}'":$PS1"

where I added a : as a separator, so not exactly what is in the question.

It is not required to use PROMPT_COMMAND. Here it does needless complications. All you have to do is define this function:

prompt_status()
{
    [ $? = 0 ] && return
    echo -n "${?} "
}

And then set PS1 like this:

shopt -s promptvars
PS1='$(prompt_status)'$PS1

Evaluation of prompt_status() inside subshell protects $? from being changed.