Find the rank of $T^2$

Let $Z_k={\rm ker} T^k$, $k\geq 0$ be the sequence of kernel spaces. One has $$Z_0 =\{0\} \subset Z_1 \subset Z_2 ...$$ Let $d_k = {\rm dim\ } Z_k$. Then one has the following inequality for $k\geq 1$: $$ d_{k+1}-d_k \leq d_k-d_{k-1}$$ In your case this yields $11-9\leq 9-d_2\;$ or $\;d_2\leq 7$ and $\;9-d_2\leq d_2-4\;$ or $\;d_2\geq 6.5$. The unique integer solution is thus $d_2=7$ and by the kernel-rank theorem we get ${\rm rank}\; T^2=4$ in accordance with the statement of Alex.

I imagine the above inequality is well-known by specialists but I don't have a reference for it. To prove it note that since $T Z_{k+1}\subset Z_k$ and $T Z_k\subset Z_{k-1}$ we have a well-defined map between quotients: $$ \widehat{T} : Z_{k+1}/Z_k \rightarrow Z_k/Z_{k-1} .$$ I claim this map is injective. If $w\in Z_{k+1} \setminus Z_k = Z_{k+1} \setminus T^{-1}(Z_{k-1})$ then the last expression shows indeed that $Tw\in Z_k\setminus Z_{k-1}$. As the map is injective dimensions must increase so $$ d_{k+1}-d_k = {\dim\;} Z_{k+1}/Z_k \leq \dim Z_k/Z_{k-1}=d_k-d_{k-1}.$$ If you don't fancy quotient spaces you may concoct a (slightly longer) proof using complements, i.e. writing $Z_{k+1} = Z_k \oplus W_k$ and look at how $T$ acts upon $W_k$.

About generality: The above inequality (showing concavity of $k\mapsto d_k$) holds in a space of any dimension (also infinite, as long as $d_1$ is finite). Thus the conclusion for $d_2$ is independent of the dimension of the ambient space. But the conclusion for the rank of $T^2$ is obviously not.


Following levap’s guide consider a $11\times 11$ complex matrix $T$ such that $\operatorname{rank} T=7$, $\operatorname{rank} T^3=2$, and $\operatorname{rank} T^4=0$. Let Jordan form of the matrix $T$ contains $a_i$ Jordan cells of size $i$ for each $1\le i\le 11$. Then we have the following system of equations.

$\begin{cases} \sum_{i=1}^{11} ia_i =11\\ \sum_{i=2}^{11} (i-1)a_i =7\\ \sum_{i=4}^{11} (i-3)a_i =2\\ \sum_{i=5}^{11} (i-4)a_i =0 \end{cases}$

It follows $a_i=0$ for $i\ge 5$ and

$\begin{cases} a_1+2a_2+3a_3+4a_4=11\\ a_2+2a_3+3a_4 =7\\ a_4=2\end{cases}$

We find consecutively from this system $a_4=2$, $a_3=0$, $a_2=1$, and $a_1=1$. So $$\operatorname{rank} T^2=\sum_{i=3}^{11} (i-2)a_i =2a_4=4.$$