Is the analysis as taught in universities in fact the analysis of definable numbers?
The concept of definable real number, although seemingly
easy to reason with at first, is actually laden with subtle
metamathematical dangers to which both your question and
the Wikipedia article to which you link fall prey. In
particular, the Wikipedia article contains a number of
fundamental errors and false claims about this concept. (Update, April 2018: The Wikipedia article, Definable real numbers, is now basically repaired and includes a link to this answer.)
The naive treatment of definability goes something like this: In many cases we can uniquely specify a real number, such as $e$ or $\pi$, by providing an exact description of that number, by providing a property that is satisfied by that number and only that number. More generally, we can uniquely specify a real number $r$ or other set-theoretic object by providing a description $\varphi$, in the formal language of set theory, say, such that $r$ is the only object satisfying $\varphi(r)$.
The naive account continues by saying that since there are only countably many such descriptions $\varphi$, but uncountably many reals, there must be reals that we cannot describe or define.
But this line of reasoning is flawed in a number of ways and ultimately incorrect. The basic problem is that the naive definition of definable number does not actually succeed as a definition. One can see the kind of problem that arises by considering ordinals, instead of reals. That is, let us suppose we have defined the concept of definable ordinal; following the same line of argument, we would seem to be led to the conclusion that there are only countably many definable ordinals, and that therefore some ordinals are not definable and thus there should be a least ordinal $\alpha$ that is not definable. But if the concept of definable ordinal were a valid set-theoretic concept, then this would constitute a definition of $\alpha$, making a contradiction. In short, the collection of definable ordinals either must exhaust all the ordinals, or else not itself be definable.
The point is that the concept of definability is a second-order concept, that only makes sense from an outside-the-universe perspective. Tarski's theorem on the non-definability of truth shows that there is no first-order definition that allows us a uniform treatment of saying that a particular particular formula $\varphi$ is true at a point $r$ and only at $r$. Thus, just knowing that there are only countably many formulas does not actually provide us with the function that maps a definition $\varphi$ to the object that it defines. Lacking such an enumeration of the definable objects, we cannot perform the diagonalization necessary to produce the non-definable object.
This way of thinking can be made completely rigorous in the following observations:
If ZFC is consistent, then there is a model of ZFC in which every real number and indeed every set-theoretic object is definable. This is true in the minimal transitive model of set theory, by observing that the collection of definable objects in that model is closed under the definable Skolem functions of $L$, and hence by Condensation collapses back to the same model, showing that in fact every object there was definable.
More generally, if $M$ is any model of ZFC+V=HOD, then the set $N$ of parameter-free definable objects of $M$ is an elementary substructure of $M$, since it is closed under the definable Skolem functions provided by the axiom V=HOD, and thus every object in $N$ is definable.
These models of set theory are pointwise definable, meaning that every object in them is definable in them by a formula. In particular, it is consistent with the axioms of set theory that EVERY real number is definable, and indeed, every set of reals, every topological space, every set-theoretic object at all is definable in these models.
- The pointwise definable models of set theory are exactly the prime models of the models of ZFC+V=HOD, and they all arise exactly in the manner I described above, as the collection of definable elements in a model of V=HOD.
In recent work (soon to be submitted for publication), Jonas Reitz, David Linetsky and I have proved the following theorem:
Theorem. Every countable model of ZFC and indeed of GBC has a forcing extension in which every set and class is definable without parameters.
In these pointwise definable models, every object is uniquely specified as the unique object satisfying a certain property. Although this is true, the models also believe that the reals are uncountable and so on, since they satisfy ZFC and this theory proves that. The models are simply not able to assemble the definability function that maps each definition to the object it defines.
And therefore neither are you able to do this in general. The claims made in both in your question and the Wikipedia page on the existence of non-definable numbers and objects, are simply unwarranted. For all you know, our set-theoretic universe is pointwise definable, and every object is uniquely specified by a property.
Update. Since this question was recently bumped to the main page by an edit to the main question, I am taking this opportunity to add a link to my very recent paper "Pointwise Definable Models of Set Theory", J. D. Hamkins, D. Linetsky, J. Reitz, which explains some of these definability issues more fully. The paper contains a generally accessible introduction, before the more technical material begins.
You can also talk about arithmetically definable real numbers: those for which the Dedekind cut of rationals is of the form: $$\{m/n: \forall x_1 \exists x_2 \ldots \forall x_{k-1} \exists x_k\, p(m,n,x_1,\ldots,x_k)=0\},$$ where the $x$'s range over integers, and $p$ is a polynomial with integer coefficients.
Then on this definition of definability: $e$ and $\pi$ and all the familiar reals are definable. Only countably many numbers are definable. There must be other real numbers which are undefinable. And it all makes sense, and is even provably consistent, in ordinary set theory.
A standard reference for this way of thinking is the system $ACA_0$ in Simpson's Subsystems of Second-Order Arithmetic.
The cost of this metamathematical simplicity is a small change to the mathematics: any definable bounded sequence of reals has a definable least upper bound, but an uncountable definable set of reals may not. Feferman's notes on Predicative Foundations of Analysis show how to develop standard analysis on this basis. If we changed mathematics as taught in universities to be based on predicative analysis, few undergraduates or people outside the math department would notice much difference.
While you cannot define undefinable numbers, you can quantify over all real numbers, whether or not they are definable. "Let $a$ be a number" does not assume that $a$ is definable, but is merely a shorthand for quantification over $a$. The theorems in analysis are safe.
Definability is a subtle issue that was only partially dealt with in Joel David Hamkins excellent answer. $(V,∈)$-definability (as a predicate on sets) is $(V,∈)$-definable if and only if every ordinal is already $(V,∈)$-definable (in which case, $(V,∈)$-definability coincides with ordinal definability; a set is $(V,∈)$-definable iff it is first order parameter-free definable in $(V,∈)$). Intuitively, not every ordinal is $(V,∈)$-definable, and this can be formalized and proved by adding a $(V,∈)$ satisfaction relation Tr and replacement axiom schema for formulas involving Tr (this is not conservative over ZFC and does not hold in the minimal transitive model of ZFC). However, every consistent theory T extending ZF has a model (called definable ordinal model or Paris model) in which every ordinal is definable; for a complete T with a well-founded model, all Paris models are well-founded. This applies even if T proves that the set of ordinal definable real numbers is countable. It also applies to theories with Tr by using $(V,∈,\mathrm{Tr})$ definability, and analogously with other extensions.
In any case, we can speak of existence of $(V,∈)$-definable examples without ambiguity since there is a $(V,∈)$-definable set satisfying $P$ iff there is an ordinal definable set satisfying $P$ (and analogously if $P$ has parameters and we allow definability with those parameters). A set is ordinal definable iff it is definable in some $V_κ$. Now, V=HOD is $Π^V_2$ conservative over ZFC (and over ZFC + $φ$ for a $Σ^V_2$ $φ$), so even if the proofs assumed definability (which they do not), the theorems (which for analysis and 'ordinary mathematics' are all $Π^V_2$) would still be correct.
That does not mean that the theorems have definable examples. In second order arithmetic (where the examples are real numbers), existence of definable examples holds assuming projective determinacy (and for $Σ^1_2$ predicates in just ZFC), but existence of ordinal definable nonmeasurable sets (and likely other 'non-well-behaved' sets of reals) is independent of ZFC and ordinary large cardinal axioms.