Finite groups of which the centralizer of each element is normal.
I believe the comment by James is correct, these groups are precisely the $2$-Engel groups.
Claim: The following statements are equivalent for a group $G$.
Every centralizer in $G$ is a normal subgroup.
Any two conjugate elements in $G$ commute, ie. $x^g x = x x^g$ for all $x, g \in G$.
$G$ is a $2$-Engel group, ie. $[[x,g],g] = 1$ for all $x, g \in G$.
Proof:
1) implies 2): $x \in C_G(x)$, thus $x^g \in C_G(x)$ since $C_G(x)$ is normal.
2) implies 3): $x^g = x[x,g]$ commutes with $x$, thus $[x,g]$ also commutes with $x$.
3) implies 1): If $[[x,g],g] = 1$ for all $g \in G$, then according to Lemma 2.2 in [*], we have $[x, [g,h]] = [[x,g],h]^2$. Therefore $[C_G(x), G] \leq C_G(x)$, which means that $C_G(x)$ is a normal subgroup.
[*] Wolfgang Kappe, Die $A$-Norm einer Gruppe, Illinois J. Math. Volume 5, Issue 2 (1961), 187-197. link
The first observation is that such a finite group is nilpotent. For, given any $x \in G,$ we certainly have $x \in F(C_{G}(x)),$ and the latter group is contained in $F(G)$ as $C_{G}(x) \lhd G.$ This essentially reduces the question to one about $p$-groups. Notice that any finite $p$-group $P$ of nilpotency class $2$ has the property (thanks to Jack Schmidt for pointing out an earlier error). I am not sure at the moment whether it is possible to characterize $p$-groups with the property in any straightforward fashion.