How to avoid duplicate entries in .bash_history

From the bash man page:

HISTCONTROL

A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not saved in the history list. A value of ignoredups causes lines matching the previous history entry to not be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.

So put the following line in your ~/.bashrc:

export HISTCONTROL=ignoreboth:erasedups

Stick this in your ~/.bashrc:

export HISTCONTROL=ignoredups

You could instead use ignoreboth. This it shorthand for both ignorespaces (commands starting with spaces) and ignoredups (duplicates).

I prefer ignoredups on its own as I find the default behaviour of ignoring commands with spaces at the front quite annoying when I copy a command off a website and it doesn't get saved because I accidentally copied in a space too.... But to each their own.

Putting this in ~/.bashrc will apply @alvin's solution across different sessions as wlell

HISTCONTROL=ignoredups:erasedups
shopt -s histappend
PROMPT_COMMAND="history -n; history -w; history -c; history -r; $PROMPT_COMMAND"

Source: Linux: Bash history: “ignoredups” and “erasedups” setting conflict with common history across sessions