Why is "lies between" a primitive notion in Hilbert's Foundations of Geometry?
The ''lies between'' notion is the basis used by Hilbert to define an ''order'' between points on a straight line. The axioms that define such a notion are called Axioms of Order in his book, and introduced thusly:
The axioms of this group define the idea expressed by the word “between,” and make possible, upon the basis of this idea, an order of sequence of the points upon a straight line, in a plane, and in space. The points of a straight line have a certain relation to one another which the word “between” serves to describe.
So obviously we can substitute these axioms with some other definition of order, as, e.g. in Birkoff's axioms, where the order is introduced via real numbers. But the importance of the Hilbert axiomatization of geometry is exactly that it does not use the real numbers.
Actually, "betweenness" can be defined from Hilbert's other primitive notions, in two steps:
1. For points $A, B, C$ say that "$A$ is closer to $B$ than to $C$", notated $AB\le AC$, iff for every line $\ell$ that contains $B$, there is a point $X$ on $\ell$ such that $AX=AC$.
2. For points $A, B, C$ say that "$B$ is between $A$ and $C$" iff the points are collinear, and $AB\le AC$ and $CB\le CA$.
Hilbert could have formulated his axioms based on these definitions (viewed as abbreviations) instead of making "between" a primitive notion. However, he probably wouldn't have considered that an improvement. He probably wasn't particularly focused on minimizing the number of primitive notions at any cost, and there would have been several costs:
The axioms themselves would be more complex to state, and less clearly true about pre-formal intuitive geometry.
It might be desirable/interesting to be able to cordon off the part of the theory that is invariant under (nonsingular) affine transformations of space -- but that can't be done if betweenness (which is an affine concept) were dependent on congruence (which isn't).
Geometry in one dimension would not be a matter of simply restricting one's attention to a single line and points on it, because the definition of $AB\le AC$ depends on the possibility that $\ell$ may be different from the line $AB$.
It would be unclear whether Hilbert's somewhat wonky "completeness" axiom (which basically asserts that we only want to consider maximal models of the other axioms) would work in this setting. At least a priori one could imagine a model that declared $A$ to be closer than $B$ than to $C$ simply because one of the lines through $B$ had gaps in it that allowed it to pass through the circle $AC$ without actually intersecting -- and this we could have deeply nonstandard models that were nonetheless maximal.
(Edit: Wikipedia's article on Tarski's geometry suggests the following simpler drop-in replacements for the definitions above, which avoid speaking about lines at all:
1. For points $A, B, C$ say that $AB\le AC$ iff for every point $Z$ such that $AZ=CZ$, there is an $X$ such that $AX=BX=CZ$.
2. For points $A, B, C$ say that "$B$ is between $A$ and $C$" iff $AX\le AB$ and $CX\le CB$ implies that $X$ must be the point $B$.
The costs above would still hold for these.)
Affine geometry can be done over an arbitrary field, including finite fields where one can't have a meaningful notion of order.
On the other hand, if one is working over the field $\mathbb{R}$ of course one has a notion of order given by the usual order on the reals.
Hilbert is interested in developing axiom systems that would describe more general frameworks that are, on the one hand, not limited to the real numbers, and on the other don't even depend on having a background field at all. To put it another way, the relation of betweenness is more general than that of order.