Example of $a,~b\in G$ such that $ab\in H\leq G$ and $a^2b^2\notin H.$

Consider $S_3$.

Let $a=(1 2 3)$ and $b=(2 3)$. Then $ab=(1 2)$ and $a^2b^2=(1 3 2)$

Let $H=\{1, ab\}$. Then $ab\in H$ but $a^2b^2\not\in H$


Take $G$ to be the free group on $a,b$, whose elements are the reduced words in the alphabet $a,b,a^{-1},b^{-1}$.

Take $H$ to be the cyclic subgroup generated by $ab$.

The non-identity elements of $H$ are the reduced words words $(ab)^n$ for $n \ge 1$ and $(b^{-1}a^{-1})^n$ for $n \ge 1$.

Since the reduced word $a^2 b^2$ does not have that form, it follows that $a^2 b^2 \not\in H$.


Let $u \in G$, $v \in H$.

Take $a=u$, $b=u^{-1}v$. Then $ab \in H$.

Moreover, $a^2b^2=uvu^{-1}v$, thus $a^2b^2 \in H \Leftrightarrow uvu^{-1}v \in H \Leftrightarrow uvu^{-1} \in H$.

Thus if $H$ is not normal, the property does not hold.