Moduli of curves over finite field
The constructions of the Deligne–Mumford stack $\mathscr M_g$ and its coarse moduli space $M_g$ are very similar, and Deligne–Mumford's original article [DM69] is surprisingly readable. Note that Deligne and Mumford write $\mathscr M_g$ (resp. $M_g$) for what is now commonly known as the moduli of stable (rather than smooth) curves $\overline{\mathscr M_g}$ (resp. $\overline{M_g}$); feel free to stick to the smooth case if you prefer.
The construction goes as follows:
If $C$ is a stable curve of genus $g \geq 2$, then $\omega_C^{\otimes 3}$ is very ample.
Construct a space $H_g \subseteq \mathbf{Hilb}_{\mathbf P^{5g-6}}$ of tricanonically embedded stable curves (using Hilbert scheme methods; the number one tool for representing moduli problems).
Construct $\mathscr M_g$ (resp. $M_g$) as the stacky (resp. GIT) quotient of $H_g$ by the automorphism action $\mathbf{PGL}(5g-6) \circlearrowright \mathbf{Hilb}_{\mathbf P^{5g-6}}$.
I just learned that there is a bit of an issue whether you take the GIT quotient first and then specialise to characteristic $p$ or vice versa. Perhaps Alper's good moduli spaces can say something about this question. Or you can use the stacky definition.
To some extent the main goal of the stacks project is to provide all details needed to define $\overline{\mathscr M_g}$, so this is another reference. But this is thousands of pages, so not really useful as an introduction.
References.
[DM69] P. Deligne and D. Mumford, The irreducibility of the space of curves of a given genus. Inst. Hautes Études Sci. Publ. Math. 36, p. 75-109 (1969). DOI: 10.1007/BF02684599.
As requested, I am posting my comment as an answer. Mumford discusses this in the preface to the first edition of "Geometric Invariant Theory".
Already in "Geometric Invariant Theory" (which was Mumford's thesis), Mumford constructed the coarse moduli scheme $M_g$ over $\text{Spec}\ \mathbb{Z}.$ You can read about what this means in Mumford's book. The great irony of GIT is that, although GIT was largely motivated by this problem, in fact Mumford constructs $M_g$ over $\text{Spec}\ \mathbb{Z}$ without using GIT (precisely because neither had Haboush yet extended GIT to char $p$, nor had Seshadri extended GIT to mixed characteristic).