How prove this $(p-1)!\left(1+\frac{1}{2}+\frac{1}{3}+\cdots+\frac{1}{p-1}\right)\equiv 0\pmod{p^2}$

The solution below is adapted from Notes on Wolstenholme’s Theorem by Timothy H. Choi.

Let $$ S=(p-1)!\sum_{k=1}^{p-1} \frac1k $$ Using your insight $$ \dfrac{1}{k}+\dfrac{1}{p-k}=\dfrac{p}{k(p-k)} $$ we have $$ 2S=(p-1)!\sum_{k=1}^{p-1} \left(\dfrac{1}{k}+\dfrac{1}{p-k}\right) = p\sum_{k=1}^{p-1} \frac{(p-1)!}{k(p-k)} = pS' $$ Note that $S'$ is an integer. Now $$ \frac{(p-1)!}{k(p-k)} \equiv (k^2)^{-1} \bmod p $$ where the inverse is taken ${}\bmod p$. This is a consequence of Wilson’s Theorem. Hence $$ S'\equiv \sum_{k=1}^{p-1} (k^2)^{-1} \equiv \sum_{k=1}^{p-1} k^2 = \frac{(p-1)p(2(p-1)+1)}{6} \equiv 0 \bmod p $$ This means that $2S\equiv 0 \bmod p^2$ and so $S\equiv 0 \bmod p^2$. (We need $p>3$ twice here.)

As others have noted, the congruence is not true for $p=3$, since $$ 2!\left(1+\frac 1 2\right)=2+1=3,$$ which is not divisible by $9$. We can still use what you suggested to prove the congruence holds $\operatorname{mod} p$. Let $p$ be an odd prime, then \begin{align*} (p-1)!\sum_{k=1}^{p-1} \frac 1 k &= (p-1)! \sum_{k=1}^{(p-1)/2} \left(\frac 1 k + \frac 1 {p-k}\right) \\&= (p-1)!\sum_{k=1}^{(p-1)/2} \frac{p}{k(p-k)} = p\sum_{k=1}^{(p-1)/2} \frac{(p-1)!}{k(p-k)}. \end{align*} Since $(p-1)!$ always contains $k$ and $(p-k)$ as a factor, the fractions in the sum are integers and the result is a multiple of $p$.

See lhf's answer for why it is even a multiple of $p^2$ as long as $p>3$.