When does a Markov chain not have a steady state?

Every Markov chain on a finite state space has an invariant distribution. As you said, this follows directly from the condition that the rows sum to $1$.

It is possible for a Markov chain on a finite state space to have multiple invariant distributions. However, the Perron-Frobenius theorem tells us that these can be decomposed into distributions which are concentrated on strongly connected components of the state space. Decomposing the process into strongly connected components results in one or more different Markov chains each of which has a unique invariant distribution. (Any transient states of the original chain will not be states in any of these sub-chains, since they are not in any strongly connected component.) In particular, an irreducible finite state Markov chain has a unique invariant distribution which assigns positive probability to all states.

However many invariant distributions the process has, it can happen that no invariant distribution is approached over time. When the state space is finite, it turns out that this only happens when the chain is "periodic" (meaning that there are states $i,j$ and an integer $n>1$ such that all paths from $i$ to $j$ have a length which is a multiple of $n$). In this case the transition matrix has an eigenvalue which is not $1$ and has modulus $1$. If the corresponding eigenvector contributes to the initial condition, then its contribution does not decay, and no invariant distribution is approached. The classic example is $P=\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$.

It is possible for a Markov chain on an infinite state space to not have any invariant distributions. This is roughly because probability mass can escape to infinity. A simple example is a Markov chain on $\mathbb{Z}$ which deterministically moves one unit to the right at every step. Another example is the simple symmetric random walk: it has an invariant measure which is uniform on $\mathbb{Z}$, but this measure cannot be normalized.


You are correct. If $P$ is a Markov transition matrix, then $P^n$ converges as $n\to\infty$ if and only if $P$'s only eigenvalue of modulus one is $\lambda=1.$