When is convolution not commutative?

Convolution of two $C_c$ functions commute $\iff$ $G$ is abelian

As you noted if $G$ is abelian then it is trivial that convolutions commute.

For the converse, let convolution of any two $C_c$ functions commute. Let $f,g \in C_c(G)$

Then $\forall x \in G \text{ we have }$ $$0= f*g(x)-g*f(x)=\int_G f(xy)g(y^{-1}) d\lambda(y) - \int_{G} g(y)f(y^{-1}x)d\lambda(y)$$ $$=\int_G f(xy^{-1})g(y)\Delta(y^{-1}) d\lambda(y) - \int_{G} g(y)f(y^{-1}x)d\lambda(y)$$ $$\implies \int_G g(y)(\Delta(y^{-1})f(xy^{-1})-f(y^{-1}x))d\lambda(y)=0$$

Since, $g \in C_c(G)$ was arbitrarily chosen, it follows that $$\Delta(y^{-1})f(xy^{-1})=f(y^{-1}x), \forall x,y \in G$$ So put $x=1$ above and note that $\Delta(y^{-1})f(y^{-1})=f(y^{-1})$ . Again $f \in C_c(G)$ was arbitrarily chosen thus $f$ can very well be non-zero at $y^{-1}$. So we get, $\Delta(y^{-1})=1, \forall y \in G$

Hence, $f(xy^{-1})=f(y^{-1}x) \forall x,y \in G$ . Then just replace $y$ by $y^{-1}$ and we get $$f(xy)=f(yx) \forall f \in C_c(G) \implies xy=yx, \forall x,y \in G$$

Since, you have the result for $C_c(G)$, it follows for $L^1(G)$


For a locally compact group $G$, one has that $L^1(G)$ is commutative if and only if $G$ is commutative. See, for example, Theorem 1.6.4 in Principles of Harmonic Analysis by Deitmar and Echterhoff. As for your desire to see an example of noncommutativity in $L^1(G)$, the aforementioned fact says that any choice of nonabelian $G$ must lead to one. A simple way to proceed is to take $G$ to be a noncommutative discrete group (or even a finite group) as, in this case, one has an inclusion $G \subset L^1(G)$. This is because each $g \in G$ may be identified with the function $\delta_g \in L^1(G)$ defined by $\delta_g(g)=1$ and $\delta_g(h)=0$ if $h \neq g$. One can check that $\delta_g * \delta _h = \delta_{gh}$. This example is not unrelated to the methods by which one would prove the equivalence between abelianness of $L^1(G)$ and $G$. The idea is to use approximations to such delta functions, or to construct a larger algebra than $L^1(G)$ (a sort of multiplier algebra) which contains them.