What's wrong with this argument that Newton's second law implies all potentials are quadratic?

While other answers are correct, they fail to address your specific issue. It looks like you are treating Newton's second law like it defines a single function, when it does not.

For example, in algebra if I say a function is $f(x)=x^2 + 3$, then I can "plug into" this function something like $sx$ so that $f(sx)=(sx)^2+3$ by how we defined the function.

This is not what Newton's second law is doing. $F(x)=m\ddot x$ is not a function saying "whatever I plug into the function $F$ I take its second derivative with respect to time and multiply it by $m$." So, your statement of $F(sx)=ms\ddot x$ is not correct. Newton's law is a differential equation, not a function definition. $F(x)$ is defined by the forces acting on our system, and Newton's second law then states that the acceleration is proportional to this force.


In order to deal with the kind of analysis you want to do, you have to be careful. It's a bit awkward to write $F(\vec{x})=m\ddot{\vec{x}}$ in the first place but you can write that as long as you understand what it means. It means that you are considering the force and acceleration both as fields because you're considering Newton's law at each point in space. So, a clearer way to write it is $$F(\vec{x})=m\ddot{\vec{x}}(\vec{x})$$


Edit $1$: Let me clarify the meaning of this expression a bit more clearly. As I said, I'm considering a particle at each point in space. So, $\ddot{x}(x)$ simply means the acceleration of the particle which is located at $x$. The $x$ is bracket is a label. For example, if I was writing down the Newton's second law for $N$ particles, I'd write $F(x_i)=\ddot{x}_i$ for $i=1,2,...,N$. Now, I put a particle at each coordinate point and the label $i$ is replaced with the coordinate label $x$. So, simply replacing $i$ with $x$ would get me $F(x(x))=\ddot{x}(x)$ where $x$ is a label just like $i$. Now, notice that $F(x(x))$ means the force at the position $x$ of a particle labeled by $x$. But the meaning of the coordinate labeling $x$, by definition, implies that the position $x$ of a particle labeled by $x$ would simply be $x$. Thus, I adopt a succinct notation for $F(x(x))$ and simply write $F(x)$. Thus, $F(x(x))=\ddot{x}(x)$ becomes $F(x)=\ddot{x}(x)$, which is the expression written above, except in vector notation.


Now, you can do the scaling game and write $$F(s\vec{x})=m\ddot{\vec{x}}(s\vec{x})$$

Now, you see that there is no reason to believe that $$\ddot{\vec{x}}(s\vec{x})=s\ddot{\vec{x}}(\vec{x})$$ in general. However, what you can do is try to see when this would be true. And if you do that, you can see that this would be true iff $$F(s\vec{x})=sF(\vec{x})$$

This is what you ultimately got. But this simply means that you've figured out the condition under which $\ddot{\vec{x}}(s\vec{x})=s\ddot{\vec{x}}(\vec{x})$ would be valid. Your mistake was that you assumed that $\ddot{\vec{x}}(s\vec{x})=s\ddot{\vec{x}}(\vec{x})$ is generically true (likely due to your confusing notation) and then concluded that $F(s\vec{x})=sF(\vec{x})$ should be true generically, which is not true because your implicit assumption is not generically true.


Edit $2$

I am considering the transformation $x\to sx$ to mean that it takes us from point $x$ to point $sx$ in the same units. So, if I'm writing Newton's law for the particle at position $x=1$ as $F_1 = a_1$, the transformation means that now I'm writing Newton's law for a different particle, one which is situated at $x=s$, and I'd write $F_s=a_s$. So nothing non-trivial is happening here. The assumption of the OP was that $a_s=sa_1$ which is a very non-trivial claim as it establishes a relation between accelerations of particles at different points. I simply point out the obvious that this is not true unless the forces at those positions are related in such a way to establish such a relation, i.e., unless $F_s=sF_1$.


What you found here is not a inconsistency of Newton's mechanics, but a symmetry of the harmonic oscillator. Consider for simplicity a point particle in $\mathbb{R}^n$. The force can be considered as a function $F:\mathbb{R}^n\rightarrow\mathbb{R}^n$ taking the position of the particle as argument. Newton's law states that a physical trajectory $$\gamma:\mathbb{R}\rightarrow\mathbb{R}^n$$ of a point-particle of mass $m$ satisfies the equation $$F(\gamma(t))=\ddot\gamma(t)$$ for all times $t\in\mathbb{R}$.

Now concerning your question, you observed that if we take a physical trajectory $\gamma$ and scale it by a real number $s\in\mathbb{R}$, this satisfies Newton's law only if $F$ is linear. However, this is no inconsistency of Newton's mechanics, since scaling of a physical trajectory in general does not give you a new physical trajectory. Instead, the proper interpretation of what you found here is that this Kind of scaling symmetry is a characteristic of the harmonic oscillator (quadratic potential).

As a conclution, you assumed, that scaling a physical trajectory gives a new physical trajectory, which is not true in general. What you found is that this symmetry is a property of linear forces/quadratic potentials. I hope this could help you! Cheers!