Does parallel transport change the subspace?

Consider the plane $\{(x, y, 0)\mid x,y \in \Bbb R\}\subset\mathbb{R}^3$, the line $t\mapsto (0, 0, t)$ and the vector field $t\mapsto(1, 1, t)$ along it. Then the parallel transport of no vector in the field along the curve will be tangent to the plane.

Negative answer. Counter-example: $(M,g) = (\Bbb R^3, \langle\cdot, \cdot \rangle)$, $N = \Bbb R^2\times \{0\}$, $\gamma(t) = (0,0,t)$ and $X(t) = (0,0,\sin t)$. Then $X(0) = 0 \in T_{\gamma(0)}N$, but the parallel transport of $X(\pi/2) = (0,0,1)$ from $\gamma(\pi/2) = (0,0,\pi/2)$ to $\gamma(0) =0$ via $\gamma$ is $(0,0,1) \not\in T_{\gamma(0)}N$. The values $X(t)$ for $t\neq t_0$ should not influence the result of transporting $X(t_0)$, since a priori one transports vectors and not fields.

We of course understand that $\dot \gamma(0) \ne 0$, lest $\gamma(t)$ be a rather trivial geodesic.

We observe that, since $\dot \gamma(0)$ is "perpendicular" to $N$, the condition

$\langle \dot \gamma(0), X(0) \rangle = 0 \tag 0$

is necessary, but not sufficient, for

$X(0) \in T_{\gamma(0)}N, \tag 1$

for (0) says that $X(0)$ has no component along $\dot \gamma(0)$; in the event that $N$ is a hypersurface in $M$, that is,

$\dim N = \dim M - 1, \tag 2$

(0) is in fact also sufficient, for then $\dot \gamma(0)$ and $T_{\gamma(0)}N$ span $T_{\gamma}(0)N$; in fact, (2) implies

$T_{\gamma(0)}M = \langle \dot \gamma(0) \rangle \oplus T_{\gamma(0)}N, \tag 3$

where $\langle \dot \gamma(0) \rangle$ is the one-dimensional subspace of $T_{\gamma(0)}M$ generated by $\dot \gamma(0)$.

Now consider any vector field $X(t)$ along $\gamma(t)$; we have

$\nabla_{\dot \gamma} \langle \dot \gamma, X \rangle = \langle \nabla_{\dot \gamma} \dot \gamma, X \rangle + \langle \dot \gamma, \nabla_{\dot \gamma} X \rangle; \tag 4$

if $X(t)$ is parallel along $\gamma(t)$, then

$\nabla_{\dot \gamma}X = 0; \tag 5$

thus (4) becomes

$\nabla_{\dot \gamma} \langle \dot \gamma, X \rangle = \langle \nabla_{\dot \gamma} \dot \gamma, X \rangle; \tag 6$

now the general formulation of the geodesic equation as I understand it is

$\nabla_{\dot \gamma} \dot \gamma = f(t) \dot \gamma; \tag 7$

here $f(t)$ is determined by the choice of the running parameter $t$ of $\gamma(t)$; if we substitute this into (6), we obtain

$\nabla_{\dot \gamma} \langle \dot \gamma, X \rangle = f(t) \langle \dot \gamma, X \rangle; \tag 8$

the solution to this linear, first order differential equation for $\langle \dot \gamma, X \rangle$ with $\langle \dot \gamma(t_0), X(t_0) \rangle = \langle \dot \gamma, X \rangle(t_0)$ is

$\langle \dot \gamma, X \rangle(t) = \exp \left (\displaystyle \int_{t_0}^t f(s) \; ds \right ) \langle \dot \gamma, X \rangle (t_0); \tag 9$

since

$\forall t \in \Bbb R, \;\exp \left (\displaystyle \int_{t_0}^t f(s) \; ds \right ) \ne 0, \tag{10}$

(9) shows that

$\langle \dot \gamma(t), X(t) \rangle = \langle \dot \gamma, X \rangle(t) = 0 \Longleftrightarrow \langle \dot \gamma, X \rangle (t_0) = \langle \dot \gamma(t_0), X(t_0) \rangle = 0; \tag{11}$

of course, if $\gamma(t)$ is affinely parametrized then

$f(t) = 0, \tag{12}$

and we in fact have

$\langle \dot \gamma, X \rangle(t) = \langle \dot \gamma, X \rangle (t_0), \; \forall t \in \Bbb R; \tag{13}$

now taking $t_0 = 0$ yields

$\langle \dot \gamma(t), X(t) \rangle = 0 \Longleftrightarrow \langle \dot \gamma(0), X(0) \rangle = 0; \tag{14}$

if we now choose $X(t)$ such that

$\langle \dot \gamma(t), X(t) \rangle \ne 0, \tag{15}$

(14) together with (0) shows the parallel transport of $X(t)$ along $\gamma(t)$ cannot satisfy

$X(0) \in T_{\gamma(0)} N, \tag{15}$

since it has a component in the direction $\dot \gamma(0)$.