Chemistry - Where does magnetic inequivalence come from?

Firstly, thank you for asking a very interesting question that certainly pushed my limits of knowledge (and others' as well). I apologise if I have made mistakes, it is a real possibility. All references are to Levitt's Spin Dynamics, 2nd ed. (an excellent book, by the way).

From a quantum mechanical point of view, the complexity in the spectrum of magnetically inequivalent nuclei should be considered the "norm". Fundamentally, in any strongly coupled system where the difference in Larmor frequency between two spins $\Delta \omega$ is much less than the coupling constant $J$, we expect to see complex spectra (see second half of this answer for more discussion).

Magnetic equivalence is a special case which leads to simplifications in the Hamiltonian and hence in the spectrum. As I mentioned in a comment, it can be shown that any coupling between magnetically equivalent nuclei "disappear". The word "disappear" merits further explanation, though. It is not true that the coupling constant $J$ goes to zero, for example. Neither is it true that it is averaged to zero by some sort of physical process, e.g. molecular rotation. It only means that the coupling term in the Hamiltonian has no impact on the expectation value of any quantum mechanical observable, or its evolution. Since an NMR spectra is simply us humans reading out some expectation values, the NMR spectrum itself is therefore unaffected by the coupling term between magnetically equivalent nuclei.

Having established that it is magnetic equivalence, not inequivalence, that is the special case, I would like to rephrase your question to: "Is there a physical explanation for why magnetic equivalence leads to simplification of the spectrum?"

The mathematical reason is given in Appendix A.9 of Levitt. I won't reproduce it here, so I will just mention it has to do with some operators which commute with others, and these only commute under very specific circumstances, i.e. the rules for magnetic equivalence that you have surely been taught (they must be chemically equivalent, plus either they have the same coupling constant to all other nuclei, or there are no other spins in the molecule).

Given the (relative) complexity of the mathematical explanation, I suspected that a physical explanation would not be particularly instructive. Levitt, though, does offer us something (p 461):

The apparent disappearance of J-couplings between magnetically equivalent spins is quite surprising. A physical explanation runs as follows. For magnetically equivalent spins, the local magnetic fields, whatever their source, are exactly the same on the two spins. It follows that the motion of the two spin polarizations is identical. Whatever happens, the polarizations of the two spins are locked in the same relative orientation. There is, therefore, no need to incorporate a term in the spin Hamiltonian that takes into account the dependence of the energy on this relative spin orientation.

What this means is: it does not matter what the relative orientations of the spins are. For example, in ethene, it does not matter whether two spins are up and two are down, or whether all four are down, or whether all four are somewhere between up and down (i.e. a superposition of $\alpha$ and $\beta$, which is in fact the most likely case by far). The relative orientations can be anything, but the point is that the time evolutions of all four spins are synchronised. Therefore, the coupling terms between them – which are terms, scalar products, that depend on the relative orientation of spin vectors – may be omitted from the Hamiltonian without any impact on the observables of the system.

Returning to a mathematical interpretation, another sign of magnetic equivalence between nuclei $i$ and $j$ is when the labels $i$ and $j$ can be permuted in the spin Hamiltonian without changing it. This implies that $[\hat{\vec{I}}_i,\hat{H}] = [\hat{\vec{I}}_j,\hat{H}]$. From the standard formula

$$\frac{\mathrm d}{\mathrm dt}\langle Q \rangle = -\frac{\mathrm i}{\hbar}\langle[\hat{Q},\hat{H}]\rangle$$

it is immediately obvious that $\mathrm d\langle\vec{I}_i\rangle/\mathrm dt = \mathrm d\langle\vec{I}_j\rangle/\mathrm dt$, thus proving that the spins $i$ and $j$ move in concert.

Regarding ethene: you are correct that the cis and trans (and vicinal) couplings in ethene are different. However, all four of the protons in ethene are chemically equivalent and must be considered together as a group. It is not correct to look at two out of the four protons and call them magnetically inequivalent, without also taking into account the remaining two. In this case, it turns out that if you do the maths, the relevant commutators don't cancel out if you only consider two out of the four nuclei. However, when you consider all four nuclei together, the relevant operators fully commute.

Now, none of the simplifications above apply to magnetically inequivalent nuclei. These experience very tiny instantaneous differences in magnetic fields – hence the name – and therefore all the couplings in the system end up being reflected in the spectrum, as we should have expected a priori. Returning to your analysis ("the coupling constants are the same, so shouldn't they be the same?"), the issue is that the coupling constant $J_\ce{AX}$ is only one part of the coupling term in the Hamiltonian, which is $2\pi J_\ce{AX}(\hat{\vec{I}}_\ce{A}\cdot\hat{\vec{I}}_\ce{X})$. Although $J_\ce{AX}$ is indeed equal to $J_\ce{A'X'}$, the dot product $(\hat{\vec{I}}_\ce{A}\cdot\hat{\vec{I}}_\ce{X})$ is in general not equal to $(\hat{\vec{I}}_\ce{A'}\cdot\hat{\vec{I}}_\ce{X'})$.

Warning: as with the ethene case, one cannot look at individual A and A' nuclei, because they are symmetry-related. You have to look at both of them together as a group. Once you do that, though, the spectra become possible to analyse.

Can magnetic inequivalence be explained without any QM?

This was the original crux of my answer, but I realised it is slightly tangential. I leave it in, however, since there is a possibility it may be useful.

The Hamiltonian of a spin system can be written as:

$$\hat{H} = \sum_i \omega_i \hat{I}_{iz} + \sum_i\sum_{j<i}2\pi J_{ij}(\hat{\vec{I}}_i\cdot\hat{\vec{I}}_j) \tag{1}$$

where $i$ ranges from $1$ to $n$, the number of spins in the system, $\omega_i$ is the Larmor frequency of spin $i$, and $J_{ij}$ is the coupling constant between nuclei $i$ and $j$ in $\pu{Hz}$ (hence the factor of $2\pi$).

Under a particular condition - when $|\omega_i - \omega_j| \gg J_{ij}$, or "weak coupling" - the term $$(\hat{\vec{I}}_i\cdot\hat{\vec{I}}_j) = \hat{I}_{ix}\hat{I}_{jx} + \hat{I}_{iy}\hat{I}_{jy} + \hat{I}_{iz}\hat{I}_{jz} \tag{2}$$ may simply be approximated by $\hat{I}_{iz}\hat{I}_{jz}$. This is called a "secular" approximation, cf. Appendix A.6 of Levitt. The reduced Hamiltonian is then

$$\hat{H} = \sum_i \omega_i \hat{I}_{iz} + \sum_i\sum_{j<i}2\pi J_{ij}(\hat{I}_{iz}\hat{I}_{jz}) \tag{3}$$

Since the Hamiltonian only consists of $\hat{I}_z$-type operators, the eigenstates of the Hamiltonian will simply be products of the eigenstates of $I_{iz}$:

$$\Psi = \psi_1\psi_2\cdots\psi_n \tag{4}$$

where $\psi_i$ is either $\alpha_i$ (up spin) or $\beta_i$ (down spin). The energies of such systems are then found by using $\langle\alpha_i|\hat{I}_{iz}|\alpha_i\rangle = +1/2$ and $\langle\beta_i|\hat{I}_{iz}|\beta_i\rangle = -1/2$. In general, therefore, the first term in the Hamiltonian will give some linear combination of $\omega_i$ (with each coefficient being $\pm 1/2$) and the second term in the Hamiltonian will give another linear combination of $2\pi J_{ij}$ (with each coefficient being $\pm 1/4$).

The allowed transitions correspond to a difference between these energies, and we regain the simple picture that we learn in introductory NMR: peaks at $\omega_i$ (or their equivalent in the chemical shift scale), and splitting between peaks of exactly $2\pi J_{ij}$ (or $J_{ij}$ in the Hz scale). Often, this is taught using a simplified model of considering each spin to be like a bar magnet, i.e. it is either up or down spin. With this model one may achieve some understanding of first-order spectra, and can rationalise the $n+1$ rule, for example.

However, this is an assumption which leads to precisely equation $(4)$. By assigning a distinct "spin" to each nucleus, we discard the possibility of quantum entanglement, i.e. we discard any possible quantum states that are not simple direct products. For example, consider the two-spin wavefunction

$$\Psi = \frac{1}{\sqrt{2}}(\alpha_1\beta_2 + \beta_1\alpha_2) \tag{5}$$

This is a perfectly valid wavefunction, but it is not possible to assign an up or down spin to either of the two nuclei. Equivalently, by assigning up or down spins to each nucleus, we are ruling out the possibility of any such states being important.

Now, this is not a problem as long as we are in the weak coupling regime. It may be helpful to think of the discarded term $(\hat{I}_{ix}\hat{I}_{jx} + \hat{I}_{iy}\hat{I}_{jy})$ as a perturbation to the simplified Hamiltonian in equation $(3)$. The unperturbed eigenfunctions of the simplified Hamiltonian are given by equation $(4)$. If the perturbation is small, i.e. if we are in the weak coupling regime, then the eigenfunctions of the full Hamiltonian are very close to that of the simplified Hamiltonian, and hence our simplified model (with discrete up and down spins) works. However, if the perturbation is large, i.e. strong coupling, then the simplified model fails. The eigenstates are no longer the same, the energy eigenvalues are modified, and hence the transition energies (which depend on the energy eigenvalues) and intensities (which depend on the eigenstates) are totally different.

Having two nuclei which are chemically equivalent, i.e. having the same chemical shift, is a case which simply not play into our simplified picture above. Since they have the same chemical shift, $|\omega_i - \omega_j| \equiv 0$ and hence weak coupling is never true. Note that chemically equivalent nuclei still possess a coupling constant $J_{ij}$ to each other and hence, in fact, chemical equivalence represents the most extreme case of strong coupling.

Fortuitously, when two chemically equivalent nuclei are also magnetically equivalent, it can be shown that the coupling is simply not manifested in the spectra (Appendix A.9, Levitt). Therefore, we can still use our simplified model, but simply with the caveat that "equivalent protons do not couple to each other". Strictly speaking the model is already incorrect, because the eigenstates of the system are no longer products of individual spin functions (see my answer here).

This is, however, not the case for magnetically inequivalent nuclei. Therefore, we have to throw our simplified model out - it just does not work here. It is therefore – to the best of my knowledge – not possible for one to explain the physical origin of magnetic inequivalence based on simplified arguments such as how "spin A sees spin B".