orthogonal projection onto a manifold

I think you do need $M$ to be closed for this to work. Consider $M$ the open unit disk (in $\mathbb{R}^2$) embedded into $\mathbb{R}^3$ like an almost closed taco, ie the points $(1,0)$ and $(-1,0)$ are mapped to the same point in $\mathbb{R}^3$ (note that they are not part of $M$). You don't get a well defined orthogonal projection around that point.

The adequate concepts to make you idea precise are those of the normal bundle and of tubular neigborhoods of a smooth submanifold $M \subset \mathbb R^s$. See for example

John M. Lee, Introduction to Smooth Manifolds (p. 139 ff)

and

https://mathoverflow.net/q/283467.