What is an observer in quantum mechanics?

Are we talking quantum mechanics? Then I'd say that a "measurement" is any operation that entangles orthogonal states of the system under consideration with orthogonal states of the environment. "Measurement" is the important thing in most formulations of QM. Colloquially speaking, an observer is something that performs measurements.

The only other place in physics I can think of where "observer" shows up is in the oft-used phrase "This is obvious to the casual observer". This is just shorthand for "I can't be bothered to write out the mathematical proof".

Either the observer is classical or the observer is quantum. If the observer is classical, we are back to the Heisenberg cut of the world into a quantum part and a classical part, and the explanatory gap needs to be bridged in this manifestly dualistic interpretation. If the observer is quantum, then another observer needs to observe the first quantum observer by the tenets of quantum mechanics. Down the road of infinite regress we go.

As long as the concept of an observer can't be made mathematically precise and unambiguous, the measurement problem will never be solved.

I prefer a variant of Anonymous Coward's answer given above, by leaving out the environment. I would say that an observer is a system that interacts with the systems it observes by entangling orthogonal states of the systems under consideration with orthogonal states of itself and possibly other systems.

So, I don't bring in the baggage of an environment here, but the possibility of that is included by mentioning "other systems". The fundamental point is that an observer is capable of extracting information from a system. A simple example is the CNOT gate which in the |0>,|1> basis acts on two qubits by applying the NOT operation on the second qubit if the first qubit is |1> and acts as the identity otherwise. This means that when the second qubit in initialized in the |0> state, it can "measure" the first qubit. The observer is thus the CNOT gate, the second qubit is its record of the observation it has made.