When is a vector "glued" to the origin?
I think it is helpful to think about these concepts on an arbitrary manifold. In that setting, there is no such thing as a position vector, because the manifold may not be a vector space; positions are specified instead by local coordinates.
The only reason we can identify positions with vectors in Euclidean space and move them around is that (1) Euclidean space is a vector space, and (2) the space carries an affine connection, which supplies a notion of parallel transport. You can't even "move around" tangent vectors on an arbitrary manifold unless you have a connection (though every Riemannian manifold comes with a natural one, the Levi-Civita connection); and in addition, as Ted says in his answer, you can't imagine those tangent vector glued "wherever you like" unless the tangent bundle is trivializable. The tangent bundle on a vector space is always trivializable.
A very useful example to keep in mind is polar coordinates in the Euclidean plane. Here a position is specified by $(r,\theta)$ or, especially in physics and engineering, a "position vector" $r\mathbf{\hat{r}}$. The former is the manifold point of view; the latter exploits the fact that we are laying curvilinear coordinates on what, at bottom, is a vector space. But you quickly learn this thing called a position vector is not really a position vector, because the vector $\mathbf{\hat{r}}$ changes from point to point. It doesn't live in the underlying space; rather, it lives in the tangent bundle, and the tangent spaces change from point to point. We are only able to think of it as a bona fide position vector because we imagine uprooting it from its tangent space and gluing it instead to the tangent space at the origin. But, to repeat, in a general manifold, the underlying space isn't necessarily a vector space and doesn't necessarily have a zero vector.
In general, to specify a position we just gives its local coordinates, and that is that: we don't need a vector to specify a position. You might wonder, then, where velocity vectors come from, because we're so used to getting velocity vectors by differentiating position vectors. But velocity vectors in arbitrary manifolds don't come from differentiating position vectors. They come from partial differentiation in directions specified by the local coordinates. More precisely, given local coordinates $(x_1, x_2)$, we represent a velocity vector as the operator $a\frac{\partial}{\partial x_1}+b\frac{\partial}{\partial x_2}$. (This is part of the reason, I think, many people struggle with the abstract definition of a tangent vector. For years they have been allowed to think of a velocity vector as the derivative of a position vector.)
You can see how the two points of view diverge when you ask a physicist or engineer to tell undergraduates how to represent velocity or acceleration vectors in planar polar coordinates. Most likely, they will draw a nice picture showing why the (time) derivative (along some curve) of $\mathbf{\hat{r}}$ is $\dot{\theta}\mathbf{\hat{\theta}}$. But the picture only works if you can parallel transport the tangent basis vectors $\mathbf{\hat{r}}$ and $\mathbf{\hat{\theta}}$ at a later point back along the curve to the tangent space at the initial point -- and it is exactly this seemingly innocent feature of the picture that requires an affine connection in general. What the calculation is really doing is exploiting the standard Euclidean connection to specify the covariant derivative / Christoffel symbols of planar polar coordinates.
Your discussion is correct, but somewhat special to $\mathbb{R}^n$. If you try to do the same thing on a sphere, you run into problems. You still have a tangent bundle with an $\mathbb{R}^2$ at each point, but you can't "move vectors around" in the same way that you can in $\mathbb{R}^n$, because any continuous vector field on a sphere must be 0 somewhere. So if you start with a nonzero vector at some point on the sphere, you can't continuously assign a corresponding nonzero vector at all points of the sphere.
More generally, a manifold in which it's possible to "move vectors around" from one tangent space to another continuously is called a parallelizable manifold (search that term for a more precise definition).
It is easy to prove that an isomorphism from one vector space $V$ to another, $W,$ must map the origin of $V$ to the origin of $W$. But for any non-origin vector $v\in V$ and any non-origin vector in $w\in W,$ there is some isomorphism that maps $v$ to $w.$ "Isomorphism" in this case means a mapping that is
- linear, and
- one-to-one, and
- onto.
One could say that a position vector is a point in an affine space and a displacement vector, which takes some position vectors to others, is a point in a vector space. In an affine space there is no privileged point which is the "origin"; in a vector space there is one. The "privileged" displacement is the zero displacement; there is no privileged position.
One can look at it like this: Suppose John Doe and Richard Roe disagree on which location in physical space should be considered the origin. They both attempt to compute a linear combination of several points. Let's say those several points are in Bennington, Vermont and Pyongyang, North Korea and the location of the International Space Station at noon today (Greenwich time). They disagree on the results because they disagree on where the origin is. Except that if the sum of the coefficients is $1,$ then the will agree despite their differing choice of origin. A linear combination in which the sum of the coefficients is $1$ is called an affine combination. So one could say that a vector space is a set of points equipped with an operation of linear combination satisfying the algebraic laws of linear combinations, and an affine space is a set of points equipped with an operation of affine combination satisfying the algebraic laws of affine combinations. And an affine space becomes a vector space as soon as you choose one of its points to be the origin.