Why does Fixed Point Iteration work?

Let us assume that function $g$ is defined on an interval $(a,b)$, $g(x)\in(a,b)$ in that interval, and that there is a constant $c<1$ such that for each $x,y \in (a,b)$, \begin{equation} \left|g(y)-g(x)\right| < c |x-y|. \tag{1} \end{equation} If $g$ has a derivative, this becomes $g'(x)<c$ for $x\in(a,b)$.

The fixed point iteration is defined by $x_{k+1}=g(x_k)$, where $x_0$ is an arbitrarily chosen starting point in $(a,b)$.

Let us assume that the function has a fixed point at $\hat{x}\in(a,b)$, that is $\hat{x}=g(\hat{x})$.

Now at step $k$, the absolute error of our current guess to the fixed point is $e_k = |x_k-\hat{x}|$. We get $$ e_{k+1} = |x_{k+1}-\hat{x}| = |g(x_k)-g(\hat{x})| < c|x_k - \hat{x}| = c e_k. $$

Therefore, the sequence $(e_k)_k$ is nonnegative and bounded above by the sequence $(c^ke_0)_k$, which converges to $0$. Therefore, $\lim_{k\to\infty}e_k=0$. This means that the fixed point iteration converges to $\hat{x}$.


For general $g:\mathbb{R}\to\mathbb{R}$, we can make following observations:

If (1) holds in $\mathbb{R}$, we can replace $(a,b)$ with $\mathbb{R}$ in the above proof. One can also see that the function has exactly one fixed point in that case (if $g$ is differentiable, the derivative of $g(x)-x$ is smaller than a negative constant, thus $g(x)-x$ has exactly one zero; if $g$ is not differentiable, a similar argument still holds).

If (1) does not hold in $\mathbb{R}$ but holds in an interval $(a,b)$ containing a fixed point, we can see that $g(a)>a$ and $g(b)<b$, so $g(x) \in (a,b)$ as required. Now the fixed point iteration converges to the fixed point if $x_0$ is chosen inside the interval.


This method of finding fixed points requires that the function under consideration fulfills some rather stringent conditions, e.g. it maps the domain of definition into itself and is contracting (which means $|f(x)-f(y)| \le C|x-y|$ for some $0<C<1$).

You seem to be under the impression that it is common that this approach will give you a solution. This is incorrect. Usually these assumptions are not fulfilled. You are just lucky if they are and then the method yields the desired result. Why this is true can be seen best by looking at a proof (by iterating k times, the constant $C$ will come into play with a power of $k$, and since $C<1$, $C^k$ tends to $0$. You will find the details in analysis textbooks, search for Banach's fixed point theorem).

Also, this method is usually (at least classically) not used to calculate the fixed point/zero, but mostly to show that such a point actually exists. This may have changed during the last decades due to the availability of computing power.