Chemistry - Is it true that all of the physical laws and mathematics needed to describe chemistry are known?

Solution 1:

First, it is one thing to know the basis for a "law", and quite another to mathematically calculate the effects of that law. Consider just carbon, for example... chains may be made of thousands of atoms, with various functional groups attached to each. Though, as Dirac stated, it helps to have "shortcuts" to computation such as Fast Fourier transforms, there are still problems that cannot be solved in a "reasonable" time.

Second, if there are unknown laws, how would we know about it (not to quote Rumsfeld on unknown unknowns)?

And finally, even if all physical laws were known and understood, it would still be impossible to predict everything: Kurt Gödel's incompleteness theorems show that in a complex system (it does not have to be very complex; basic grammar-school mathematics qualifies), questions can be asked that cannot be proved true or false. This extends to chemistry and physics.

Solution 2:

This sounds an awful lot like something I said at a talk this week, so I feel obligated to answer.

First, in terms of fundamental interactions, yes, excluding a quantum theory of gravity we have a quantum field theory for how the other fundamental forces work (electromagnetic/weak and strong).

@DavePhD mentions that Dirac was wrong at least up to the development of QED. This is true. Dirac could write the non-relativistic molecular Hamiltonian down. He knew that even if he couldn't solve it, all the physics were still there, and so in principle the system was "knowable." This is just like how we can't solve the gravitational many-body problem exactly but we definitely know how Newtonian gravity works.

Anyway, fast forward to today. We can write the QED equations of motion down which account for the time-evolution of all field operators. In principle this contains all the interactions necessary to describe molecules. Computational complexity aside, I think it is important to recognize that we must get rid of several degrees of freedom before we can even define a molecule. Most importantly we have to remove the possibility of electron-positron pair creation, since in QED particle number is not constant. Molecules, after all, are defined (in our chemists' minds) as having a fixed number of electrons. Heck, even "particles" themselves are not really present in QED, they are just excitations of the underlying quantum field.

We do something similar even in non-relativistic QM, where we fix the geometry of molecules via the Born Oppenheimer approximation (and treat nuclei as classical). If we didn't, the Schrodinger equation would describe every possible geometry of a collection of atoms and electrons (Well, geometry is not well defined in a electron-nuclear wave function, but you get the idea).

All this to say, writing the equations that govern a molecule will never be as simple as just "writing down the fundamental interactions", and I think Dirac got that wrong. Approximations will always be necessary as long as we hold onto a conception of a molecule as a fundamental object of study.

Molecular QED today is limited to effective Hamiltonian theories involving photon absorption and emission. But most of the same results can be derived from a classical view of EM fields.

Most of QED is unnecessary for an accurate description of chemistry. Pair creation processes are in energy ranges that we just don't access in the laboratory. The one area which it could excel is relativistic electronic structure theory. Our current attempts are based off of the Dirac equation, which really only holds for one spin-$1/2$ particle. Given the extension to multiple particles, we have to resort to approximate relativistic treatments. The most accurate -- in the sense that it contains the most physics -- relativistic two electron interaction term I've come across is the Breit interaction, but even that is an approximate electron-electron repulsion term. We don't know the exact relativistic term within the structure of relativistic electronic structure theory. But that's okay for now, since even including the Breit term is overkill for most molecular systems.

As far as not knowing all the fundamental interactions relevant for chemistry, let me finish with one example that I find particularly fascinating, and that is the nature of chirality of molecules. One area of study is whether or not one enantiomer of a chiral molecule is energetically more stable than the other. Even if there is a slight difference, over long periods of time it may explain why life tended to evolve using L-amino acids, among other things. This energy difference is hypothesized to be very small, on the order of 10$^{-11}$ J / mol.

Anyway, this theoretical difference in energy of chiral molecules cannot be explained using any theory of electromagnetic interactions, because the EM interactions are identical in chiral molecules. Instead, the difference in energy (if there is one) must come from a parity-violating term, which shows up only in the electroweak interaction. So this area of study is known as electroweak chemistry. As far as I know, the exact form of this parity violating term is up for debate (and there may be multiple terms), since it necessarily has to couple to some sort of magnetic perturbation, like spin-orbit coupling. Because no one really knows exactly what this term looks like, it makes it difficult for theorists to predict what the possible energy difference between enantiomers should be. Which then makes it very difficult for spectroscopists to know what to probe for.

So this is an example of a fundamental interaction relevant to chemistry (albeit a small one), that we don't really know much about, but would give huge insights into the evolution of life as we know it.


Solution 3:

Dirac is probably right but, even if he isn't, it probably isn't important for chemistry.

The issue highlighted by Dirac is that, even if we do understand all the relevant laws of quantum mechanics as far as they determine chemical properties, that doesn't help us turn chemistry into a branch of mathematics. The problem is that while we understand the equations we don't have good ways of solving those equations except for the simples systems. For example, we only have exact solutions to the electron wave function equations (which is what determines most chemical properties) for the simplest possible atoms (one nucleus, one electron). Everything else is an approximation.

This should not be a surprise. The three body problem for Newtonian gravity has no exact solutions (or, more strictly, only a very small number for some very special cases). Quantum wave functions are more complex than that and systems with multiple electrons are not going to have neat mathematical solutions.

What this implies is that we can't reliably predict the chemical properties of anything more complicated than hydrogen atoms from the physical laws they obey even if we completely understand the laws. We can approximate but it is hard to tell if reality deviates from the rules because our approximations are poor or because we don't understand some detail of the rules.

So even if there is some subtle detail of the laws we don't understand, it would be hard to verify the implications for chemistry.

There might be a few areas where obscure parts of quantum mechanics do impact chemistry (though speculation is all we have right now because of the limitation described above). Normally in the quantum mechanics used for chemistry, we are just looking at electromagnetic forces, and that is complicated enough. Some people have speculated that other forces might have small influences that matter for chemistry. For example, there is speculation that some interaction with nuclear forces might explain life's preference for single optical isomers in many living structures. The speculation suggests that optical isomers have very slightly different energies because of a tiny interaction with the asymmetry in non-electromagnetic forces (see this example). But these effects are, if they exist, small compared to the uncertainty in our predictions based on the well-known laws.

So the dominant problem is chemistry is the quality of our approximations not the potential existence of entirely new laws.


Solution 4:

Dirac published that statement in Quantum Mechanics of Many Electron Systems, received 12 March 1929.

In 1948 Verwey and Overbeek demonstrated experimentally that the London dispersion interaction is even weaker than 1/$r^6$ at long distance (say hundreds of Angstroms or more).

Casimir and Polder soon thereafter explained with quantum electrodynamics (QED) that the dependence should be 1/$r^7$ at relatively long distances.

See section 3.1.1 Van der Waals Forces, in Some Vacuum QED Effects

So Dirac was wrong at least up to the development of QED.


Solution 5:

As already pointed out in a comment by John Custer, Dirac was 100% correct in

The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble. It therefore becomes desirable that approximate practical methods of applying quantum mechanics should be developed, which can lead to an explanation of the main features of complex atomic systems without too much computation.

For light systems, typically thought as H-Kr, non-relativistic quantum mechanics i.e. the Schrödinger equation is sufficient to describe chemistry; for heavier nuclei you need to use relativistic quantum mechanics i.e. the Dirac equation which is a bit more complicated. In many cases we can invoke the Born-Oppenheimer method, and assume that the nuclei are moving in the instantaneous field of the electrons; now all we need to solve is the electronic problem.

We know that the exact solution to the electronic problem in this case is achievable with the full configuration interaction (FCI) a.k.a. exact diagonalization method, in which you describe the electronic many-particle wave function by a weighted sum of electronic configurations a.k.a. Slater determinants as $|\Psi \rangle = \sum_k c_k |\Phi_k\rangle$. These electronic configurations are built by distributing $N$ electrons into $K$ single-particle states a.k.a. orbitals. For the method to be accurate, $K\gg N$, and actually to get the exact solution you need $K \to \infty$.

Now, to find the ground state (as well as any excited states), you just need to diagonalize the many-electron Hamiltonian in the basis of the electron configurations. But, the problem is that the number of the electron configurations grows extremely rapidly.

If we assume that we are looking at a spin singlet state, then you have $N/2$ spin-up and $N/2$ spin-down electrons. For each spin, there are ${K \choose N/2} = \frac {K!} {\frac N 2 ! (K-\frac N 2)!} $ ways to populate the orbitals. The total number of electron configurations for $N$ electrons in $K$ orbitals, typically denoted as ($N$e,$K$o), is then ${K \choose N/2}^2 = \left[ \frac {K!} {\frac N 2 ! (K-\frac N 2)!} \right]^2$.

Even for the case of a very small number of orbitals, $K=N$, the number of configurations quickly becomes huge. (8e,8o) has 4900 configurations, (10e,10o) has 63 504, (12e,12o) has 853 776, (14e,14o) has 11 778 624, (16e,16o) has 165 636 900, (18e,18o) has 2 363 904 400, (20e,20o) has 34 134 779 536, and (22e,22o) has 497 634 306 624.

Although you can still solve the (8e,8o) problem with dense matrix algebra on modern computers, you see that very quickly you have to become very smart in how to diagonalize the matrix. Because the Hamiltonian is a two-particle operator, it is extremely sparse in the basis of the electron configurations: if two configurations differ by the occupation of more than two orbitals, the Hamiltonian matrix element is zero by Slater and Condon's rules. Moreover, for large problem sizes you also want to avoid storing the matrix, which is why you want to use an iterative method. (The famous Davidson method for iterative diagonalization was actually developed exactly for this purpose!)

With smart algorithms, billion-configuration calculations i.e. the (18e,18o) problem have been possible since the early 1990s, see e.g. Chem. Phys. Lett. 169, 463 (1990). However, despite the huge increase in computational power in the last 30 years, the barrier has barely budged: as far as I am aware, the largest FCI problem that has been solved is the (22e,22o) calculation in J. Chem. Phys. 147, 184111 (2017).

The thing to note here is that even the (22e,22o) calculation is not large enough to solve a single atom in an exact fashion: you need a lot more orbitals to achieve quantitative accuracy with experiment. Although a high-lying orbital contributes only very little to the correlation energy, there are A LOT of them.

Exactly as Dirac wrote, approximations are needed. Density-functional approximations are extremely popular in applications, but they are far from exact. On the other hand, high-accuracy studies often employ the coupled-cluster method, which is a reparametrization of the FCI method; however, it, too would exhibit exponential scaling were it not truncated - i.e. approximated.

Tags: