Law of large numbers for a sequence of random variables

Assuming uniform integrability of $X^m$, we have that $\mathrm E[X^m]\to \mathrm E[X]$, $m\to\infty$. Therefore, it is enough to show that $$ \left| (\overline{X^m})_m - \mathrm E[X^m]\right|\to 0, m\to \infty,\tag{1} $$ almost surely, where $(\overline{X^m})_m = \frac1m \sum_{i=1}^m X_i^m$.

One possibility is to go through concentration inequalities. For example, if the variables are bounded, as in your question, then by the Hoeffding inequality, for any $\varepsilon>0$, $$ \mathrm P\left(\left| (\overline{X^m})_m - \mathrm E[X^m]\right|>\varepsilon\right)\le e^{-C \varepsilon^2 m} $$ with some $C>0$. Using the Borel-Cantelli lemma, we easily get $(1)$.

Another possibility is, as I commented, to deduce the uniform convergence $$ \sup_m \left| (\overline{X^m})_n - \mathrm E[X^m]\right|\to 0, n\to \infty,\tag{2} $$ from the uniform law of large numbers. However, it seems unlikely that the almost sure convergence can be shown this way; I will only outline the convergence in probability.

Let $F^m$ be the cdf of $X^m$ and $Q^m(t) = \sup\{x\in \mathbb R: F^m(x)<t\}, t\in(0,1)$, be its quasi-inverse (quantile function). Then, as it is well known, $X^m \overset{d}{=} Q^m(U)$, where $U$ is a uniform $[0,1]$ variable. Therefore, $$ (\overline{X^m})_n \overset{d}{=} \frac1n \sum_{k=1}^n Q^m(U_k), $$ where $U_1,U_2,\dots$ are iid uniform $[0,1]$ variables. Also it follows from the weak convergence of $X^m\to X^0$ that $Q^m\to Q^0$ pointwise in the continuity points of $Q^0$, hence, almost everywhere on $(0,1)$.

Now let $\Theta = \{m^{-1}, m\ge 1\}\cup \{0\}$ and set $f(t,m^{-1}) = Q^m(t)$, $m\ge 1$, $f(t,0) = Q^0(t)$. Then, as is explained above, $f(t,\theta)$ is continuous in $\theta$ for almost all $t$ (modulo the distribution of $U$). Therefore, assuming existence of integrable majorant of $f(U,m^{-1})=Q^m(U)$ (which is easily seen to be equivalent to uniform integrability of $X^m$), we get that $$ \sup_{\theta\in \Theta}\left| \frac1n \sum_{k=1}^n f(U_k,\theta) - \mathrm{E}[f(U,\theta)]\right| \to 0, n\to \infty, $$ almost surely, whence we get the convergence $(2)$ in probability (remember that we replaced $(\overline{X^m})_n$ by its distributional copy).

The convergence in probability might sound bad, but there are at least two advantages:

  1. Only uniform integrability is required.

  2. The approach works for any $(n_m,m\ge 1)$ such that $n_m\to\infty$, $m\to\infty$, i.e. we have $$ \left| (\overline{X^m})_{n_m} - \mathrm E[X^m]\right|\to 0, m\to \infty, $$ in probability. The first approach fails (to establish the almost sure convergence) for "small" $n_m$.