What constitutes a symmetry for Noether's Theorem?
By a symmetry is meant an off-shell symmetry. An on-shell symmetry is a vacuous notion, as OP has already noted. See also this related Phys.SE post.
Noether's theorem may be generalized to quasi-symmetries, cf. my Phys.SE answers here & here.
A symmetry of a classical theory described by an action $S[\varphi]$ where $\varphi$ is the set of all fields in the theory is a field redefinition $\varphi(x) \to \varphi'(x)$ such that $S[\varphi] = S[\varphi']$. Note that the field redefinition is such that it does not act on the coordinates $x$.$^\dagger$ Note also that this definition of a symmetry is off-shell.
The statement of Noether's theorem is that for every a connected (to the identity) continuous off-shell symmetry of an action, there exists a current $j_\mu(x)$ that is conserved on-shell. Thus, even though the symmetry exists at an off-shell level, the current is only conserved on-shell.
A symmetry of a quantum theory is a field redefinition $\varphi(x) \to \varphi'(x)$ such that the path integral measure is invariant $$ [d\varphi]e^{-S[\varphi]} = [d\varphi']e^{-S[\varphi']} $$ In this case, the statement of Noether's theorem (or in this case, known as Ward Identity) is $$ \partial^\mu \langle j_\mu(x) {\cal O}_1(x_1) \cdots {\cal O}_n(x_n) \rangle = 0 \quad \text{if}~x\neq x_1,x_2,\cdots,x_n~. $$ Strictly speaking, there is no notion of on-shell or off-shell in a quantum theory, but one could loosely speaking, say that all correlators are "on-shell" in the sense that correlators of fields at distinct points satisfy the equations of motion. One could also derive a more general Ward identity which tells us what happens when $x=x_i$ for some $i\in\{1,\cdots,n\}$, but I will not do that here.
$^\dagger$This is crucial and often a something that a lot of people confuse with. In field theories, all symmetry transformations act only on the fields, not on the coordinates. One often like to talk about spacetime symmetries which are described as acting on coordinates in some way $x \to x'$. However, it is crucial to remember that that is simply a tool to package information about how fields transform. For instance, you might like to talk about translations. This is described by the field redefinition $\phi(x) \to \phi'(x)$ where $\phi'(x+a) = \phi(x)$. Note that the equation $\phi'(x+a) = \phi(x)$ is to be understood as a way to deduce what is $\phi'(x)$ in terms of $\phi(x)$ and not as translations acting on the coordinates in some way.
The source of your confusion comes from the definition of symmetry of action. When Noether's theorem says that A symmetry operation is the one which remains the action invariant it does not mean $\delta S = 0$ in the sense of finding classical trajectories. Remember the proof of Noether's theorem uses the classical Euler-Lagrange equation. That means the calculation is done on shell. Here the invariance of action is examined for the particular symmetry operation. That includes External symmetry like $x -> x' = \lambda x$ or Internal Symmetry such as $\phi -> \phi'$. So you see the variation of action with respect to these variations. If the action is invariant then you call it a symmetry. On the other hand $\delta S = 0$ means variation of action w.r.t its parameters and setting them to zero.