If $x \in R$ is non-invertible implies $x^2 \in \{\pm x\}$ and $|R| >9$ odd then $R$ is a field

Firstly, observe this: If $F_3$ is the field of three elements, the ring $R=F_3\times F_3$ has $9$ elements, and its nonunits are precisely of the form $(\alpha, 0)$ or $(0, \alpha)$ for $\alpha\in F_3$, and elements of this form manifestly still satisfy the condition $x^2\in\{-x,x\}$. And it is not a field. This is an exceptional case which is why $n=4$ was left out.

Now, note that the condition implies $x^3=x$ for all nonunits. That precludes the existence of any nonzero nilpotent elements, so by the Artin-Wedderburn theorem$^\ast$ this ring is a finite product of finite fields. A different way to see this is that in a finite ring the prime ideals must be maximal, and only finitely many. With the Chinese Remainder Theorem, you can conclude, since there are no nonzero nilpotent elements, that $R$ is isomorphic to a finite product of $R/M_i$ where $M_i$ are maximal ideals.

Suppose for a moment there was more than one ring in the product.

Each factor field is a subset completely made up of nonunits, and therefore must satisfy $x^3-x=0$. Since they are each fields, that means each one is either $F_2$ or $F_3$. None can be $F_2$ because that would make the order of the ring even.

What eliminates a product of more than two $F_3$'s? Well, the thing is that you can't get $x^2=x$ or $x^2=-x$ for all the coordinates at once. For example, $(1,2,0\ldots)^2=(1,1,0\ldots)$ which is neither $(1,2,0\ldots)$ nor $(-1, -2,0\ldots)$.

So the ring has to be a field.


$^\ast$ The Artin-Wedderburn theorem states that a right Artinian ring with Jacobson radical zero is a finite product of matrix rings over division rings. A finite ring is of course Artinian. Since the Jacobson radical of a right Artinian ring is nilpotent, its elements are nilpotent elements, so ruling out nonzero nilpotent elements makes the Jacobson radical zero. Finally, commutativity rules out the matrix rings having dimension anything other than $1$. That's why you are only left with a finite product of finite fields in this case.


There is also a way to do this using elementary methods, without using Artin-Wedderburn.

We denote $R^*$ the invertible [by multiplication] elements in $R$. We say that an element $a$ is invertible if precisely $a \in R^*$. We call an element $c$ in $R$ a scalar if $c$ can be written $c=1+1+ \ldots +1$ [i.e., $c$ is obtained if one adds up $1 \in R$ precisely $m$ times for some positive integer $m$]. Let $x_1,x_2,\ldots,x_r \in R$ we define the set Span$\{x_1,x_2,\ldots,x_r\}$ to be the set of elements $\alpha \in R^*$ of the form $\alpha = \sum_{i=1}^r c_ix_i$ where the $c_i$s are scalars in $R$.

Claim 0: Let $R$ be as per the conditions of this problem, and let us assume that $R$ is not a field. In $R$ the scalars 1 and 2 is invertible, but 3 is 0. So in other words the scalars are isomorphic to $\mathbb{F}_3$.

Indeed, let $x$ be any nonzero element in $R \setminus R^*$. Let us assume that $x^2+x=0$. Then $x(x+1) =0$ so $x+1$ is not invertible either. Thus by the comments here $3x$ and $3(x+1)$ are both 0, and so this implies that $3$ also must be $0$. A similar line of reasoning would hold if $x^2-x=0$. However as per the comments, the element $2 \in R$ is an invertible element in $R$. So Claim 0 follows.

Claim 1: Let $b \in R$ but not be a scalar. Then one of $\{b,b+1,b+2\}$ is in $R^*$.

Claim 1 follows because one can use Claim 0 to show that $(b+i)^2$ cannot be in $\{(b+i),-(b+i)\}$ for all $i=0,1,2$, unless $b$ is itself a scalar. $\surd$

Claim 2: Let $a$ is in $R^*$. Then $a^2=1$.

Proof of Claim 2: Indeed, let $x$ be any nonzero nonscalar element in $R \setminus R^*$. Let us assume that $x^2+x=0$. Then $x(x+1) =0$ so $x+1$ is not invertible either, and thus, $ax$ and $a(x+1)$ are nonzero [as $a$ is invertible] and not invertible either. On the one hand $[a(x+1)]^3 = a^3[x+1]^3 = a^3[x+1]$ and on the other hand, $[a(x+1)]^3 = a[x+1]$. [Here I am using the result in rschwieb's answer that says $y^3=y$ for all $y \in R \setminus R^*$] Likewise $[ax]^3 = a^3x= ax$. So subtracting the equation $a^3x=ax$ from $a^3[x+1]=a[x+1]$ gives $a^3=a$. As $a$ is in $R^*$ one can multiply both sides of $a^{-1}$ to get Claim 2.

If in fact $x$ is a nonzero nonscalar noninvertible element satisfying $x^2-x=0$ instead, then $x(x-1)=0$ so both $x$, $x-1$ are noninvertible. The above line of reasoning still holds. Thus Claim 2 follows here as well. $\surd$

Claim 3: Let $x$ be a nonzero nonscalar element in $R \setminus R^*$. Then if $x^2+x=0$ then both $x$ and $x+1$ are not invertible and every element $b \in R$ is in Span$\{x,x+1\}$. If $x^2-x=0$ then both $x$ and $x-1$ are not invertible and every element $b \in R$ is in Span$\{x,x-1\}$.

Proof of Claim 3: Let us assume for now that $x^2+x=0$. Then $x(x+1)=0$ and so $x+1$ is not invertible either. Furthermore, $(x+1)^2=x+1$. Also, let $b$ be an element not in Span$\{x,x+1\}$. Then $b+1$ and $b+2$ are not in Span$\{x,x+1\}$ as well. Let us use Claim 1 then and let $a \in \{b,b+1,b+2\}$ be an invertible element; then $a$ is not in Span$\{x,x+1\}$.

Now on the one hand $[ax]^2 = a^2x^2 = -x$ [by Claim 2 $a^2=1$ and $x^2=-x$ by assumption]. On the other hand, as $ax$ is nonzero noninvertible as well, $[ax]^2 \in \{ax,-ax\}$. So $-x \in \{ax,-ax\}$. Similarly, $[a(x+1)]^2 = a^2(x+1)^2 = x+1$, and also, $[a(x+1)]^2 \in \{a(x+1),-a(x+1)\}$, so $(x+1) \in \{a(x+1),-a(x+1)\}$. This gives us 4 cases; $-x \in \{ax,-ax\}$, and $(x+1) \in \{a(x+1),-a(x+1)\}$. We work through 2 of these cases here:

Case 3.1: $-x=ax$, and $(x+1) = a(x+1)$. Then adding these equations together yields $1=2ax+a$, which plugging in $-x=ax$ gives $1=-2x+a$ which gives $a \in$ Span$\{x,x+1\}$ after all, which is a contradiction.

Case 3.2: $-x=ax$ and $(x+1)=-a(x+1)$. Then adding these together gives $1=-a$ which gives $a \in$ Span$\{x,x+1\}$ after all, which is a contradiction.

One can handle the remaining two cases in a similar fashion as Cases 3.1 and 3.2 to show that $a$ must be in Span$\{x,x+1\}$ here as well if $x$ is a noninvertible nonscalar nonzero element satisfying $x^2+x=0$. If $x$ is a noninvertible nonscalar nonzero element satisfying $x^2-x=0$ instead, then one can use the above line of reasoning again to show that $a$ so defined must be in Span$\{x,x-1\}$ with both $x,x-1$ noninvertible. So Claim 3 follows. $\surd$

Claim 4: Let $R$ be as in the problem and suppose that there is a $x$ that is nonzero and in in $R \setminus R^*$. Then $R$ has precisely 9 elements.

As shown above in Claim 3, if $x^2+x=0$ then $R \subseteq$ Span$\{x,x+1\}$. As $3x =0$ for all noninvertible $x \in R$ and as both $x$ and $x+1$ are not invertible it follows that $R$ has at most 9 elements. However, it is easy to see that any ring with 3,5, or 7 elements must be a field so $R$ must have exactly 9 elements.

Similarly if $x^2-x=0$.

And the result follows from Claim 4. $\surd$