Suppose $A,B\in M_n(\mathbb{C})$ such that $AB-BA=A $ . Prove that $A$ is not invertible .

Let $A$ be invertible.

We have $(AB-BA)A^{-1}=I$, or $ABA^{-1}-B=I.$

But $\operatorname{tr} (ABA^{-1})=\operatorname{tr}{B},$ which is a contradiction.

It is probably worth showing that $A$ is in fact nilpotent. Indeed, by induction we get $$A^m B - B A^m = m A^m$$ for all $m\ge 0$. Taking the traces on both sides we get $$0=m \operatorname{Trace}A^m$$ so (assuming char $0$) $$\operatorname{Trace}A^m = 0$$ for all $m\ge 1$. This implies $A$ nilponent.

$\bf{Added:}$ Based on an idea of @Hans: , we can generalize this . For $X$, $Y$ matrices, let $[X,Y]=X Y - Y X$.

Assume that $[C,B]=A$, and $[C,A]=0 $ ( $C$, $A$ commute). Then $A$ nilponent.

Indeed, for all $m\ge 1$ we have $[C,A^{m-1} B] = A^{m-1} [C,B]= A^m$, and so $\operatorname{Trace} A^m=0$ . Conclude $A$ nilpotent (assume char $0$).

$\def\m{\mathfrak }$This result (and, in fact, orangeskid's observation) has the following far reaching generalization:

if $\m g$ is a finite dimensional (complex) Lie algebra and $\mathfrak r$ is its radical, then $[\m g,\m r]$ acts on any finite dimensional representation of $\m g$ nilpotently.

Indeed, in the question the matrices $A$ and $B$ span a Lie algebra $\m g$ of dimension (at most, really, but let us suppose) equal to $2$ which is solvable, so that the radical of $\m g$ is simply $\m g$ itself, and the theorem above tells us that $[\m g,\m g]$ acts nilpotently on finite dimensinal modules: since $[\m g,\m g]$ is spanned by $A$, the desired result follows.

Of course, this is immensely more general.

As an example:

if $A$, $B$ and $C$ are square matrices such that $CA-AC = B$, $CB-BC = aA + B$, $AB-BA = 0$, then $B$ is nilpotent and, more generally, al linear combinations of $A$ and $B$.

This comes from looking at a random 3-dimensional solvable Lie algebra.