What is so interesting about the zeroes of the Riemann $\zeta$ function?

Short answer: Understanding the distribution of the prime numbers is directly related to understanding the zeros of the Riemann Zeta Function.**

Long Answer: The prime counting function is defined as $\pi(x)=\sum_{p\leq x} 1,$ which counts the number of primes less than $x$. Usually we consider its weighted modification $$\psi(x)=\sum_{p^{m}\leq x}\log p$$ where we are also counting the prime powers. It is not hard to show that $$\pi(x)=\frac{\psi(x)}{\log x}\left(1+O\left(\frac{1}{\log x}\right)\right),$$ which means that these two functions differ by about a factor of $\log x$.

The prime number theorem states that $\psi(x)\sim x$, but this is quite hard to show. It was first conjectured by Legendre in 1797, but took almost 100 years to prove, finally being resolved in 1896 by Hadamard and de la Vallée Poussin. In 1859 Riemann outlined a proof, and gave a remarkable identity which changed how people thought about counting primes. He showed that (more or less) $$\psi(x)=x-\sum_{\rho:\zeta(\rho)=0}\frac{x^{\rho}}{\rho}-\frac{\zeta^{'}(0)}{\zeta(0)},$$ where the sum is taken over all the zeros of the zeta function. ${}^{++}$

Notice that this is an equality. The left hand side is a step function, and on the right hand side, somehow, the zeros of the zeta function conspire at exactly the prime numbers to make that sum jump. (It is an infinite series whose convergence is not uniform) If you remember only 1 thing from this answer, make it the above explicit formula.

An equivalence to RH: Current methods allow us to prove that $$\psi(x)=x+O\left(xe^{-c\sqrt{\log x}}\right).$$ This error term decreases faster then $\frac{x}{(\log x)^A}$ for any $A$, but increases faster then $x^{1-\delta}$ for any small $\delta>0$. In particular, proving that the error term was of the form $O\left(x^{1-\delta}\right)$ for some $\delta>0$ would be an enormous breakthrough. The Riemann Hypothesis is equivalent to showing the error term is like square root $x$, that is proving the statement $$\psi(x)=x+O\left(x^{\frac{1}{2}}\log^{2}x\right).$$ In other words, the Riemann Hypothesis is equivalent to improving the error term when counting the prime numbers.

Remark: In your question you incorrectly state the Riemann Hypothesis, which says that all zeros have real part $\frac{1}{2}$. The fact that infinitely many zeros lie on the line was shown by Hardy in 1917, and in 1942 Selberg showed that a positive proportion lie on the line. In 1974 Levinson showed that this proportion was at least $\frac{1}{3}$, and Conrey 1989 improved this to $\frac{2}{5}$.

** Of course, there may be some people who are interested in the zeros of the zeta function for other reasons. Historically the prime numbers are what first motivated the study of the zeros.

${}^{++}$: Usually the trivial zeros will be separated out of the sum, but I do not make this distinction here. Also, Riemann's original paper states things in terms of $\Pi(x)$ and $\text{li}(x)$, the Riemann pi function and logarithmic integral, rather then $\psi(x)$. This is a very slight difference, and I use $\psi(x)$ above because it is easier and cleaner to do so.

See also: Why is $\zeta(1+it) \neq 0$ equivalent to the prime number theorem?

The initial interest in the zeros is their connection with the distribution of primes, which is often done via asymptotic statements about the prime counting function. In analytic number theory, it is standard fare to have an arithmetic function defined by a summation formula, and then modify it into a form that is easier to manipulate and obtain results for, in such a way that the asymptotic results about the modified function can be translated into results about the original function very easily. This is certainly the case with $\pi(x)$, which is why I bring this up. Most of the information that is relevant here can be found on the Explicit formula article at Wikipedia, for explicit formulas for the $\pi(x)$ function using the zeros of the Riemann zeta function. Two key highlights:

$(1)$ "This formula says that the zeros of the Riemann zeta function control the oscillations of primes around their 'expected' positions."

$(2)$ "Roughly speaking, the explicit formula says the Fourier transform of the zeros of the zeta function is the set of prime powers plus some elementary factors."

With the very basics of complex numbers we see that $x^\rho$, as a function of $x$, has a magnitude given by $x^{\Re (\rho)}$ and its argument by $\Im(\rho)\cdot\log x$. The imaginary parts thus contribute oscillatory behavior to the explicit formulas, while the real parts say which imaginary parts dominate over others and by how much - this is some meaning behind the 'Fourier transform' description. Indeed, given the dominant term in an asymptote for $\pi$ we have roughly the primes' "expected positions" (we are taking some license in referring to positioning when we are actually speaking of distribution in the limit), and the outside terms will speak to how much $\pi$ deviates from the expected dominant term as we take $x$ higher and higher in value. If one of the real parts differed from the others, it would privilege some deviation over others, changing our view of the regularity in the primes' distribution.

Eventually it also became clear that more and more results in number theory - even very accessible results that belie how deep the Riemann Hypothesis really has come to be - were equivalent to or could only be proven on the assumption of RH. See for example here or here or here. I'm not sure if any truly comprehensive list of the consequences or equivalences actually exists!

Moreover it is clear now that RH is not an isolated phenomenon, and instead exists as a piece in a much bigger puzzle (at least as I see it). The $\zeta$ function is a trivial case of a Dirichlet $L$-function as well a case of a trivial case of a Dedekind $\zeta$ function, and there is respectively a Generalized Riemann Hypothesis (GRH) and Extended Riemann Hypothesis for these two more general classes of functions. There are numerous analogues to the zeta function and RH too - many of these have already gained more ground or already had the analogous RH proven!

It is now wondered what the appropriate definition of an $L$-function "should" be, that is, morally speaking - specifically it must have some analytic features and of course a functional equation involving a reflection, gamma function, weight, conductor etc. but the precise recipe we need to create a slick theory is not yet known. (Disclaimer: this paragraph comes from memory of reading something a long time ago that I cannot figure out how to find again to check. Derp.)

Finally, there is the spectral interpretation of the zeta zeros that has arisen. There is the Hilbert-Pólya conjecture. As the Wikipedia entry describes it,

In a letter to Andrew Odlyzko, dated January 3, 1982, George Pólya said that while he was in Göttingen around 1912 to 1914 he was asked by Edmund Landau for a physical reason that the Riemann hypothesis should be true, and suggested that this would be the case if the imaginary parts of the zeros of the Riemann zeta function corresponded to eigenvalues of an unbounded self adjoint operator.

This has spurred quantum-mechanical approaches to the Riemann Hypothesis. Moreover, we now have serious empirical evidence of a connection between the zeta zeros and random matrix theory, specifically that their pair-correlation matches that of Gaussian Unitary Ensembles (GUEs)...

The year: 1972. The scene: Afternoon tea in Fuld Hall at the Institute for Advanced Study. The camera pans around the Common Room, passing by several Princetonians in tweeds and corduroys, then zooms in on Hugh Montgomery, boyish Midwestern number theorist with sideburns. He has just been introduced to Freeman Dyson, dapper British physicist.

Dyson: So tell me, Montgomery, what have you been up to?
Montgomery: Well, lately I've been looking into the distribution of the zeros of the Riemann zeta function.
Dyson: Yes? And?
Montgomery: It seems the two-point correlations go as... (turning to write on a nearby blackboard): $$1-\left(\frac{\sin\pi x}{\pi x}\right)^2$$
Dyson: Extraordinary! Do you realize that's the pair-correlation function for the eigenvalues of a random Hermitian matrix?

(Source: The Spectrum of Riemannium.)

If so inclined one can see the empirical evidence in pretty pictures e.g. here.

Here is a visual supplement to Eric's answer, based on this paper by Riesel and Göhl, and Mathematica code by Stan Wagon:

The animation demonstrates the eventual transformation from Riemann's famed approximation to the prime counting function

$$R(x)=\sum_{k=1}^\infty \frac{\mu(k)}{k} \mathrm{li}(\sqrt[k]{x})=1+\sum_{k=1}^\infty \frac{(\log\,x)^k}{k\,k!\zeta(k+1)}$$

to the actual prime-counting function $\pi(x)$, through a series of successive corrections based on the nontrivial roots of $\zeta(s)$. (Here, $\mu(k)$ is the Möbius function and $\mathrm{li}(x)$ is the logarithmic integral.) See the Riesel/Göhl paper for more details.