Why is the ground state important in condensed matter physics?
To add to Vadim's answer, the ground state is interesting because it tells us what the system will do at low temperature, where the quantum effects are usually strongest (which is why you're bothering with QM in the first place). OR it is interesting because the finite temperature behavior can be treated as a perturbation above the ground state.
For example, in a metal, the dividing line between "low" and "high" temperature might be the Fermi temperature (essentially the temperature that is equivalent to the highest occupied electron state). For many metals the Fermi temperature is on the order of $10^4 K$ or more, so a metal at room temperature is nearly in its ground state, with a few excitations given by Fermi-Dirac statistics.
As another example, if you consider a permanent magnet, the relevant temperature scale is the Curie temperature which might be hundreds of K, so a room temperature magnet could be considered to be in its ground state with some excitations (perturbations) on top of that.
Ground state contains information about most thermodynamic properties of the system at zero temperature. In fact, it can be thought of as a limiting case of the partition function at zero temperature. In many respects many physical systems never depart far from their ground state (although this notably not the case when dealing with phase transitions).
Obviously, there are many problems - notably all kinds of dynamical problems, such as relaxation or transport phenomena - that cannot be reduced to studying ground state.
It is also worth pointing out the ambiguity of the language: we are not literally interested in the state or the absolute value of its energy, but rather how this state and its energy come about from various types of interactions, and how they depend on the parameters.
Update
Here is my answer to another question that underscores the special role of the ground state in optics.
The equilibrium properties at low-enough temperatures (for metals at room temperatures, $k_BT\ll E_F$ where $E_F$ is the Fermi energy) can be determined by knowing the properties of the ground state.
At any temperature, $\mathrm{T}$, the equilibrium state of a system is dictated by the minimization of its Helmholtz free energy: $$F=U-T S\tag{1}$$ where $$U=\sum_{n} p_{n} E_{n} \quad{\rm where}\quad p_{n}=\frac{\exp \left(-\beta E_{n}\right)}{\sum_{n} \exp \left(-\beta E_{n}\right)}.\tag{2}$$ At sufficiently low temperatures, it is the minimization of $U$ that essentially determines the equilibrium state. Moreover, at low $\mathrm{T}, U$ can be well-approximated by the ground state energy $E_{0}$: $$U \approx E_{0}\tag{3}.$$ Equilibrium configuration can be determined by knowing these states.