How to create a button that deletes a character?

Button["Backspace", 
 SelectionMove[EvaluationNotebook[], Previous, Character];
 SelectionMove[EvaluationNotebook[], All, Character];
 NotebookDelete[EvaluationNotebook[]]]

This almost works:

Button[
 "Backspace",
 SelectionMove[EvaluationNotebook[], Previous, Character];
 SelectionMove[EvaluationNotebook[], All, Character];
 NotebookApply[EvaluationNotebook[], ""]
 ]

The only problem, as far as I know, with this solution is that if the cursor is at the beginning of the line then it will delete the character after instead of before it:

I would use a front end token for this purpose:

Button["Backspace", FrontEndTokenExecute["DeletePrevious"]]

This approach doesn't delete anything when the cursor is at the beginning of the input field.

Short animation: