Why can't every quantum state be expressed as a density matrix/operator?

The statement by yuggib is correct. To put it in perspective, I'll start with a completely general formulation, and then I'll show how vector-states and density operators fit into that picture. I won't try to be mathematically rigorous here, but I'll try to give an overview with enough keywords and references to enable further study.

State = normalized positive linear functional

Every quantum state, pure or mixed, can be represented by a normalized positive linear functional on the operator algebra. Such a functional takes any operator $X$ as input and returns a single complex number $\rho(X)$ as output, with nice properties like \begin{gather*} \rho(X+Y)=\rho(X)+\rho(Y) \hskip2cm \rho(cX)=c\rho(X) \\ \rho(X^*X)\geq 0 \hskip2cm \rho(1)=1 \end{gather*} for all operators $X,Y$ and complex numbers $c$. I'm using an asterisk both for complex conjugation and for the operator adjoint, and I'm writing $1$ both for the identity operator and for the unit number. I'm also considering only bounded operators to keep the statements simple. This is always sufficient in principle, even though we normally use some unbounded operators in practice because it's convenient.

"Normalized positive linear functional" is a long name for a very simple thing. It also has a shorter name: mathematicians often just call it a state (see Wikipedia), and I'll use that name here. In [1], it is called an algebraic state to distinguish it from other usages of the word "state."

A state is called mixed if it can be written as $$ \rho(X) = \lambda_1\rho_1(X)+\lambda_2\rho_2(X) $$ for all $X\in{\cal A}$, where $\rho_n$ are two distinct states and where the coefficients $\lambda_n$ are both positive real numbers (not zero). A state that cannot be written this way is called pure.

This is all completely general. It works just fine in everything from a single-qubit system to quantum field theory. In contrast, using a density operator to represent a state is mathematically less general. The following paragraphs address how vector-states and density matrices fit into the more general picture described above.

Vector states and density matrices / density operators

The GNS theorem says that a state can always be implemented as $$ \rho(\cdots) =\frac{\langle\psi|\cdots|\psi\rangle}{ \langle\psi|\psi\rangle} $$ where $|\psi\rangle$ is a single vector in some Hilbert-space representation of the operator algebra. Even mixed states can always be implemented this way. The catch is that the required Hilbert-space representation is not necessarily irreducible, and we may need to switch to different Hilbert-space representations to implement different states this way. The Hilbert-space representation of the operator algebra is irreducible if and only if the state is pure [2][3].

A state $\rho$ is called a normal state if an operator $\hat\rho$ (a density matrix or density operator) exists such that [4] $$ \rho(\cdots) = \text{Trace}(\cdots \hat \rho). $$ The fact that this kind of state has a special name suggests that it is a special kind of state — that not every state can be expressed this way. This is confirmed in [5], where counterexamples are described by Valter Moretti. The Math SE question [6] also asks for a counterexample, and it has an answer.


This is all consistent with yuggib's statement

not every quantum state can be represented, in a given (irreducible) representation, as a ray in Hilbert space (or as a density matrix, actually).

The statement needs to be parsed carefully, though: the words given and irreducible are important. The Wikipedia page that said "Describing a quantum state by its density matrix is a fully general alternative..." might be referring to a less-general context, like finite-dimensional Hilbert spaces, or might be implicitly using a less-general definition of "state." That doesn't mean the Wikipedia page is wrong; it just means that — as always — we need to beware of equivocation.


[1] Valter Moretti (2013), Spectral Theory and Quantum Mechanics (A 2018 edition is also available; I cited the 2013 version because it's the one I had on hand when writing this answer)

[2] Proposition 1.8 in https://arxiv.org/abs/math-ph/0602036

[3] Theorem 14.12 in [1]

[4] https://ncatlab.org/nlab/show/state+on+a+star-algebra

[5] Is there a physical significance to non-normal states of the algebra of observables? (on Physics SE)

[6] "Non normal state" (https://math.stackexchange.com/q/2962163)

Addendum: This answer has been downvoted a couple of times. I don't know why (no comments were left), but I'm adding the following clarification in case it addresses the concern:

If the question had been "Are normal states sufficient for all practical purposes?" then the answer would surely be yes. But that wasn't the question. The quesetion asked for the reason behind a specific mathematically-minded statement about states on operator algebras, and that's what this answer tries to address.