Number of elements of order $p$ in a (possibly infinite) group

We know that the result is true for finite groups (by the group-action argument you're familiar with).

But if there's only finitely many elements of order $p$, we can look at $H$ the group generated by those elements. It is finite, and contains all the elements of order $p$. So the result holds for $H$, and since $H$ has all the elements of order $p$, the result holds for $G$.

---

I hope this helps ^_^

Added by Derek Holt: proof that $H$ is finite. Since $H$ is generated by the elements of order $p$ and there are only finitely many, they have only finitely many conjugates, so their centralizers have finite index in $H$. So the intersection of their centralizers, which is the centre $Z(H)$ of $H$ has finite index in $H$.

Now, by a result of Schur, the derived group $[H,H]$ is finite. Since $H/[H,H]$ is an abelian group generated by finitely many elements of order $p$, it is also finite, and hence $H$ is finite.

Suppose $G$ is a group with finitely many elements of order $p$. We may assume that $G$ is generated by its elements of order $p$ (otherwise, just look at the subgroup they generate). Since every conjugate of an element of order $p$ has order $p$, each element of order $p$ must have centralizer of finite index. So, the intersection $Z$ of the centralizers of elements of order $p$ is finite index. Since $G$ is generated by its elements of order $p$, $Z$ is actually the center of $G$.

So, the center of $G$ has finite index. This implies that the commutator subgroup $[G,G]$ is finite (see this question for instance). But $G/[G,G]$ is an abelian group generated by finitely many elements of finite order, so it is finite as well. Thus $G$ is finite, and so the result for finite groups applies.