How does gravity work underground?
The other answers provide a first-order approximation, assuming uniform density (though Adam Zalcman's does allude to deviations from linearity). (Summary: All the mass farther away from the center cancels out, and gravity decreases linearly with depth from 1 g at the surface to zero at the center.)
But in fact, the Earth's core is substantially more dense than the outer layers (mantle and crust), and gravity actually increases a bit as you descend, reaching a maximum at the boundary between the outer core and the lower mantle. Within the core, it rapidly drops to zero as you approach the center, where the planet's entire mass is exerting a gravitational pull from all directions.
The Wikipedia article on "gravity of Earth" goes into the details, including this graph:
"PREM" in the figure refers to the Preliminary Reference Earth Model.
Larger versions of the graph can be seen here
And there are other, smaller, effects as well. The Earth's rotation results in a smaller effective gravity near the equator, the equatorial bulge that results from that rotation also has a small effect, and mass concentrations have local effects.
Assuming spherically symmetric mass distribution within Earth, one can compute gravitational field inside the planet using Gauss' law for gravity. One consequence of the law is that while computing the gravitational field at a distance r < R (with R being the radius of the Earth), one can ignore all the mass outside the radius r from the center
\begin{equation} \oint_{S_r} g_r \cdot dA = -G \int_{B_r} \rho dV \end{equation}
where gr is the gravitational field at distance r from Earth's center, ρ is Earth's density, Sr is the sphere of radius r centered on Earth's center of mass and Br is the volume enclosed by Sr. Assuming that ρ only depends on the distance r from the center of the Earth, we can simplify this as follows
\begin{equation} \oint_{S_r} g_r \cdot dA = -4\pi G \int_0^r \rho(s) ~s^2ds \end{equation}
\begin{equation} g_r = -\frac{G}{r^2} \int_0^r \rho(s)~s^2ds \end{equation}
Setting Mr to denote the portion of Earth's mass enclosed within Sr, we can rewrite the last formula as
\begin{equation} g_r = -\frac{GM_r}{r^2} \end{equation}
Now, letting ρr denote the average density of the portion of the Earth enclosed within Sr, we have
\begin{equation} g_r = -\frac{4 \pi G \rho_r r}{3} \end{equation}
The conclusion is that the gravity inside Earth depends roughly linearly on the distance from the center of the planet and density variations account for the deviations from linearity.
An interesting way to visualize this is to think of an over 12,700 kms long elevator from Hamilton, New Zealand to Cordoba, Spain. During the travel (which at average speed of 200km/h would take almost three days) passengers would feel gradual and roughly linear decrease in weight and in the middle of the journey would experience weightlessness followed by gradual increase in weight as they near the surface on the other side. Also, around the midpoint of the journey the floor and the ceiling would swap.
Acceleration due to gravity at depth d below the earth's surface is given by:
$g(d) = G M_e \dfrac{R_e - d}{R_e^3}$
Where,
G = Universal gravitational constant
Me = Mass of the earth
Re = Radius of the earth
d = depth below the earth's surface
