Why is general relativity background independent and electromagnetism is background dependent?
I believe this to be a very important conceptual novelty of GR. Let me explain.
Electromagnetism depends on the background
Consider the action functional for the Maxwell's theory of the electromagnetic field: $$ S[A] = - \frac{1}{4} \int d^4 x\, \sqrt{-g} F_{\mu \nu} F^{\mu \nu}. $$
This action depends on the electromagnetic potential $A_{\mu}(x)$, as well as on the metric tensor $g_{\mu \nu}(x)$. However, $A_{\mu}(x)$ appears in the square brackets of $S[A]$ and $g_{\mu \nu}(x)$ doesn't.
This actually makes all the difference. We say that $A$ is a dynamical variable, while $g$ is an external parameter, the background. Maxwell's theory is thus dependent on the exact background geometry, which is usually taken to be Minkowski flat spacetime.
Gravity is background independent
Now General Relativity is given by a diffeomorphism-invariant integral $$ S[g] = \frac{1}{16 \pi G} \intop_{M} d^4 x \sqrt{-g} \,R_{\mu \nu} g^{\mu \nu}. $$
Note the profound difference: now $g$ appears in the square brackets of $S[g]$! Spacetime geometry is a dynamical variable in gravity, just as the electromagnetic field is a dynamical variable in Maxwell's theory.
We no longer have an external background spacetime, because spacetime is dynamical. The (classical) spacetime metric tensor is not given a-priori, but is a solution of Einstein's equations. That's background independence.
Diff invariance vs background independence
These concepts are very different.
Diffeomorphism invariance simply means that we are building a field theory on a spacetime manifold, and any coordinate patch is equally good for describing the theory. Note how both the Maxwell (for electromagnetism) and Einstein-Hilbert (for gravity) actions in this post are written in the diffeomorphism-invariant form.
Background independence is basically whether a theory depends on an external spacetime geometry or not. Maxwell action depends on external background geometry, while Einstein-Hilbert (and its high-energy modifications) doesn't. In simple words, it is about whether $g$ is in the square brackets of $S$.
Why should we care
First, background independent physics is very different from the "old" physics on the background. The absense of the timelike Killing vector field renders generally ill-defined such concepts as time and energy.
Second, there's a physical insight of background independence: space and time aren't some static arena inhabited by fields, but rather have the same dynamical properties that fields have.
Third, there is no place for evolution in external time in the background independent context, because there is no external time. The implications are far-reaching: no general energy conservation (except in particular solutions), no Hamiltonian, no unitarity in the quantum theory. This is known as the problem of time. This doesn't indicate that background independent theories are unphysical, however, just that we have to utilize completely different techniques in order to derive predictions. E.g. the background independent dynamics is described in terms of constraints.
Making electromagnetism background independent
It is really easy to make Maxwell's theory background independent. We just have to couple it to gravity:
$$ S[A,g] = \intop_{\mathcal{M}} d^4 x \sqrt{-g} \left( \frac{1}{16 \pi G} R_{\mu \nu} g^{\mu \nu} - \frac{1}{4} F_{\mu \nu} F^{\mu \nu} \right). $$
Anything coupled to General Relativity (or its high-energy modification) is background independent, because $g$ appears in the square brackets of the total action.
Short conclusion
Electromagnetism is a theory of an (electromagnetic) field in spacetime. Gravity is a theory of spacetime itself.
Why is it really difficult to formulate a ToE
The difficulty in formulating a ToE lies in two completely different issues:
- Gravity is hard to consistently quantize. This is partially related to background independence. Manifestly background independent quantization procedures exist, e.g. Loop Quantum Gravity.
- It is hard to establish a natural unification of gravity and Standard Model (unless you believe in compactifications). This has little to do with background independence, mostly being an issue of obtaining a complex gravity+gauge Lagrangian from some simple geometrical model.