If $A$ is $4\times2$ and $B$ is $2\times4$, prove that $AB$ is not invertible

It is true that $AB$ is not invertible, but your reasoning is not correct. To see that there must be an error in your reasoning, try to find an example of matrices $A$ and $B$ as in your question for which the $2 * 2$-matrix $BA$ is invertible. This is doable.

In order to see why $AB$ is not invertible it helps to think about the linear maps the matrices encode. In order to be invertible the map corresponding to $AB$ must be injective and surjective. But think about what happens to a vector in 4-dimensional space that get fed into $AB$. First it is squeezed into a 2-dimensional space by $B$ (which causes a problem with injectivity), and then the points of this two-dimensional space are mapped into a much bigger 4 dimensional space by $A$, which causes a problem with surjectivity.

I am sure you can fill in the details in this argument.


By the rank-nullity theorem we get $\dim ker(B) \ge 2.$ Thus there is $x \in ker(B)$ with $x \ne 0$, then $ABx=0$. This shows that $AB$ is not invertible.


Your proof isn't correct, as pointed out by Eric.

The point is that the rank of $AB$ is at most 2 :
Say we are working with matrices with coefficients from a field $K$. $A$ represents a morphism $\varphi : K^4 \mapsto K^2$. Hence, $rank(A) = dim(\operatorname{Im}(\varphi)) \leqslant 2$. Say $B$ represents a morphism $\psi$. The rank of $AB$ is $dim(\operatorname{Im}(\varphi \circ \psi)) \leqslant dim(\operatorname{Im}(\varphi))\leqslant 2$. If $AB$ were invertible, it would have rank 4.