What does the dipole moment really represent?
For a globally-charged system
If your system has a nonzero net charge $Q$, then the quantity $$ \mathbf r_\mathrm{COC} = \frac{1}{Q} \int \mathbf r\; \rho(\mathbf r)\mathrm d\mathbf r \tag 1 $$ defines the center of charge of the distribution.
How is this concept useful? Well, in short, if your charge distribution is localized and you're fairly far away from it, then it's reasonable to try to approximate its electrostatic potential as a point-charge source, i.e., $$ V(\mathbf r) \approx \frac{Q}{4\pi\epsilon_0} \frac{1}{|\mathbf r-\mathbf r_0|}, \tag 2 $$ where $\mathbf r_0$ is the position of the putative point charge. The center of charge $\mathbf r_\mathrm{COC}$ in $(1)$ is the optimal position for this $\mathbf r_0$, i.e., the approximation $(2)$ works best when $\mathbf r_0 = \mathbf r_\mathrm{COC}$.
For a neutral system
For a neutral system, on the other hand, two things happen:
- The definition $(1)$ for $\mathbf r_\mathrm{COC}$ becomes undefined, since $Q=0$.
- The approximation $(2)$ becomes meaningless, since it just returns $V(\mathbf r) \approx 0$. And, while the potential is generally "small" in some sense, it's generally never exactly zero -- and the approximation $(2)$, however, true, is definitely not useful.
So, what are we to do? The answer comes from understanding where the approximation $(2)$ comes from. In short, the full potential is known to be of the form $$ V(\mathbf r) = \frac{1}{4\pi\epsilon_0} \int \frac{\rho(\mathbf r')}{|\mathbf r-\mathbf r'|}\mathrm d\mathbf r', \tag 3 $$ and the approximation $(2)$ comes from a Taylor expansion of the Coulomb kernel, $\frac{1}{|\mathbf r-\mathbf r'|}$, in powers of $(\mathbf r' - \mathbf r_0)/|\mathbf r-\mathbf r_0|$ (where $\mathbf r_0$ is an arbitrarily-chosen origin, and $\mathbf r$ should be "outside" the charge distribution, i.e., further away from $\mathbf r_0$ than any of the $\mathbf r'$). As it turns out, the first couple of terms of this expansion are reasonably simple to work out: $$ \frac{1}{|\mathbf r-\mathbf r'|} = \frac{1}{|\mathbf r-\mathbf r_0|} + \frac{(\mathbf r-\mathbf r_0)\cdot (\mathbf r'-\mathbf r_0)}{|\mathbf r-\mathbf r_0|^3} + \mathcal O\mathopen{}\left(\frac{|\mathbf r'-\mathbf r_0|^2}{|\mathbf r-\mathbf r_0|^3}\right)\mathclose{}. \tag 4 $$ If you then plug this $(4)$ back into $(3)$, you basically get back $(2)$, with an additional correction: $$ V(\mathbf r) \approx \frac{Q}{4\pi\epsilon_0} \frac{1}{|\mathbf r-\mathbf r_0|} + \frac{1}{4\pi\epsilon_0} \frac{\mathbf r-\mathbf r_0}{|\mathbf r-\mathbf r_0|^3} \cdot \int(\mathbf r'-\mathbf r_0)\rho(\mathbf r')\mathrm d\mathbf r'. \tag{5} $$ The coefficient in that last term, $$ \mathbf d = \int(\mathbf r'-\mathbf r_0)\rho(\mathbf r')\mathrm d\mathbf r', \tag 6 $$ is the dipole moment (relative to the origin $\mathbf r_0$), so we can reexpress $(5)$ in a cleaner form: $$ V(\mathbf r) \approx \frac{Q}{4\pi\epsilon_0} \frac{1}{|\mathbf r-\mathbf r_0|} + \frac{1}{4\pi\epsilon_0} \mathbf d\cdot \frac{\mathbf r-\mathbf r_0}{|\mathbf r-\mathbf r_0|^3}. \tag{7} $$
For one thing, this provides a proof for the assertion I made earlier about globally-charged systems: since $\mathbf d = \int\mathbf r'\rho(\mathbf r')\mathrm d\mathbf r'-Q\mathbf r_0 $, it's straightforward to show that $\mathbf d$ vanishes if $\mathbf r_0 = \mathbf r_\mathrm{COC}$, and therefore so does the correction.
On the other hand, for a neutral system, we know that $Q=0$, which means that the first term in $(7)$ vanishes (as we already knew). However, we're now better equipped than before, because we now have a sub-leading term that can provide us a useful approximation: $$ V(\mathbf r) \approx \frac{1}{4\pi\epsilon_0} \mathbf d\cdot \frac{\mathbf r-\mathbf r_0}{|\mathbf r-\mathbf r_0|^3}. \tag{8} $$ Moreover, as a bonus, the relation $\mathbf d = \int\mathbf r'\rho(\mathbf r')\mathrm d\mathbf r'-Q\mathbf r_0 $ tells us that $\mathbf d$ is independent of the choice of origin $\mathbf r_0$, since $Q=0$.
(Here, you might wonder about the $\mathbf r_0$s remaining in $(8)$ $-$ can this position be optimized to make $(8)$ better? funny you should ask!)
I won't go on for longer because this is getting quite long, but that's the general idea. As you've already noted, the full framework is that of multipolar expansions. In short, they basically allow us to write the electrostatic potential $(3)$ (which is a hassle $-$ a full new integral for every $\mathbf r$!) as a series of terms of the form $$ V(\mathbf r) = \frac{1}{4\pi\epsilon_0} \sum_{\ell=0}^\infty \frac{\mathrm{poly}_\ell(\mathbf Q_\ell,\mathbf r)}{|\mathbf r|^{\ell+1}}, \tag 9 $$ where $\mathrm{poly}_\ell(\mathbf Q_\ell,\mathbf r)$ is a homogeneous polynomial in the coordinates of $\mathbf r$ of degree $\ell$, and where the coefficients of that polynomial, the $\mathbf Q_\ell$ (known as the multipole moments), are integrals of the form $$ Q_{\ell} = \int \mathrm{poly}_{\ell}(\mathbf r'-\mathbf r_0)\rho(\mathbf r')\mathrm d\mathbf r'. \tag{10} $$ This expansion is useful on many counts, but to start with, it has the key property that the factor of $|\mathbf r|^{-(\ell+1)}$ ensures that the various terms become less and less relevant with increasing $\ell$, so if we fix $|\mathbf r|$ and our tolerance for approximation, we can truncate the series in terms of just a few integrals to calculate.
More intuitively, the multipole moments $(10)$ capture more and more detail about the angular shape of the charge distribution as $\ell$ grows, and this in turn provides more and more detail about the angular shape of $V(\mathbf r)$ $-$ and, moreover, our calculation shows that the higher-$\ell$ terms with higher detail will decay faster as you get away from the distribution.
We can split the charge density into the positive and negative regions:
$$\rho = \rho_+ - \rho_-,$$
where $\rho_+$ and $\rho_-$ are non-negative functions. By doing this, the dipole moment integral splits into two,
$$\mathbf{p} = \int \mathbf{r} \rho_+(\mathbf{r})\, d^3\mathbf{r} - \int \mathbf{r} \rho_-(\mathbf{r})\, d^3\mathbf{r}.$$
We can then interpret it as the separation vector between the center of positive charge and the center of negative charge.
Dipole and higher order moments represent the dis-balance of charge in the charge distribution, even though the net charge may be zero. This is somewhat similar to the moment of inertia, as opposed to mass, or to the moments of a probability distribution (in this latter case all the moments taken together provide the same information as knowing the distribution function).