How is Hessian tensor on Riemannian manifold related to the Hessian matrix from calculus?

The relation between the Riemannian Hessian and the Hessian in Euclidean space is very simple - they are the same. More precisely, the Euclidean Hessian is a particular case of the more general Riemannian Hessian.

Let $(x_1,\ldots,x_n)$ be local coordinates on a neighborhood in a Riemannian manifold $M$, and let $\Gamma_{ij}^k$ denote the Christoffel symbols of the Levi-Civita connection with respect to these coordinates. Let us massage your definition for the Hessian; we consider the vector fields $$X=X^i\frac{\partial}{\partial x_i},\;Y=Y^i\frac{\partial}{\partial x_i}.$$ Then $$\begin{align}X(Y(f))-df(\nabla_XY)&=X^i\frac{\partial}{\partial x_i}\left(Y^j\frac{\partial f}{\partial x_j}\right)-df\left(\nabla_{X^i\frac{\partial}{\partial x_i}}Y^j\frac{\partial}{\partial x_j}\right)\\&=X^i\left(\frac{\partial Y^j}{\partial x_i}\frac{\partial f}{\partial x_j}+Y^j\frac{\partial^2f}{\partial x_i\partial x_j}\right)-df\left(X^i\left(\frac{\partial Y^j}{\partial x_i}\frac{\partial}{\partial x_j}+Y^j\Gamma_{ij}^k\frac{\partial}{\partial x_k}\right)\right)\\&=X^iY^j\frac{\partial^2f}{\partial x_i\partial x_j}-X^iY^j\Gamma_{ij}^k\frac{\partial f}{\partial x_k}.\end{align}$$Now, the Christoffel symbols of the Levi-Civita connection with respect to the usual coordinates on $\mathbb{R}^n$ are all zero. Hence, the Euclidean Hessian matrix of the function $f$ is just the matrix whose $(ij)$ entry is $$\mathrm{Hess}(f)_{ij}=\mathrm{Hess}(f)\left(\frac{\partial}{\partial x_i},\frac{\partial}{\partial x_j}\right),$$where the right hand side is the Riemannian Hessian.

That Hessian matrix is useful for analyzing the behaviour of critical points of $f$, where $df_p = 0$. More precisely, let $M$ be a smooth manifold and $\nabla$ be a linear connection in $TM$. If $f\colon M \to \Bbb R$ is a smooth map, $p \in M$ is a critical point of $f$, and $(x^i)_{i=1}^n$ is a system of coordinates around $p$, we have the local expression $${\rm Hess}\,f_p(\partial_i\big|_p,\partial_j\big|_p) = \partial_i\big|_p(\partial_j f) - \require{cancel} \cancelto{0}{df_p(\nabla_{\partial_i}\partial_j)} = \frac{\partial^2f}{\partial x^i\partial x^j}(p),$$and so $${\rm Hess}\,f_p = \sum_{i,j=1}^n \frac{\partial^2f}{\partial x^i\partial x^j}(p) \,dx^i\big|_p\otimes dx^j\big|_p$$

At a generic point $p$, the bilinear map ${\rm Hess}\,f_p$ depends on the choice of connection (because of the $\nabla$ in the second term). But if $p$ is a critical point, then the map ${\rm Hess}\, f_p$ becomes independent of the choice of connection. If $p$ is not critical, one usually picks $\nabla$ to be the Levi-Civita connection of some Riemannian metric in $M$. But a priori, you can use any connection (which can lead to some awkward things, such as the Hessian not being symmetric if $\nabla$ has torsion).