Relation between the Dirac Algebra and the Lorentz group

It's pretty annoying that P&S just give you $$S^{\mu \nu} = \frac{i}{4} [\gamma^{\mu},\gamma^{\nu}]$$ from thin air, here is a way to derive it similar to Bjorken-Drell's derivation (who start from the Dirac equation) but from the Clifford algebra directly, assuming that products of the gamma matrices form a basis. Given a Clifford algebra of $\gamma^{\mu}$'s satisfying \begin{align} \{ \gamma^{\mu} , \gamma^{\mu} \} = 2 \eta^{\mu \nu} I \end{align} we note that for an invertible transformation $S$ we have \begin{align} 2 \eta^{\mu \nu} I &= 2 \eta^{\mu \nu} S^{-1} S \\ &= S^{-1}(2 \eta^{\mu \nu}) S \\ &= S^{-1}\{ \gamma^{\mu} , \gamma^{\mu} \} S \\ &= \{ S^{-1} \gamma^{\mu} S, S^{-1}\gamma^{\mu} S \} \\ &= \{ \gamma'^{\mu} , \gamma'^{\mu} \} \end{align} showing us that the Clifford algebra of matrices $$\gamma'^{\mu} = S^{-1} \gamma^{\mu} S$$ also satisfies the Clifford algebra, hence any set of matrices satisfying the Clifford algebra can be obtained from a given set $\gamma^{\mu}$ using a non-singular transformation $S$. Since the anti-commutation relations involve the metric $\eta_{\mu \nu}$, and we know the metric is left invariant under Lorentz transformations $$\eta^{\mu \nu} = \Lambda^{\mu} \, _{\rho} \Lambda^{\nu} \, _{\sigma} \eta^{\rho \sigma} $$ this immediately implies \begin{align} 2 \eta^{\mu \nu} I &= \Lambda^{\mu} \, _{\rho} \Lambda^{\nu} \, _{\sigma} 2 \eta^{\rho \sigma} I \\ &= \Lambda^{\mu} \, _{\rho} \Lambda^{\nu} \, _{\sigma} \{ \gamma^{\rho} , \gamma^{\sigma} \} \\ &= \{ \Lambda^{\mu} \, _{\rho} \gamma^{\rho} , \Lambda^{\nu} \, _{\sigma} \gamma^{\sigma} \} \\ &= \{ \gamma'^{\mu} , \gamma'^{\mu} \} \end{align} which shows that the Lorentz transformation of a gamma matrix also satisfies the Clifford algebra, and so is itself a gamma matrix, and hence can be expressed in terms of some non-singular transformation $S$ \begin{align} \gamma'^{\mu} &= \Lambda^{\mu} \, _{\nu} \gamma^{\nu} \\ &= S^{-1} \gamma^{\mu} S \end{align} where $S$ is to be determined. Since the operators $S$ represent performing a Lorentz transformation on $\gamma^{\mu}$, and Lorentz transformations on fields expand as $I - \frac{i}{2}\omega_{\mu \nu} M^{\mu \nu}$, we expand $\Lambda$ and $S$ as \begin{align} \Lambda^{\mu} \, _{\nu} &= \delta ^{\mu} \, _{\nu} + \omega^{\mu} \, _{\nu} \\ S &= I - \frac{i}{2} \omega_{\mu \nu} \Sigma^{\mu \nu} \end{align} where $\Sigma^{\mu \nu}$ must be anti-symmetric and constructed from a basis of gamma matrices, hence from \begin{align} \gamma'^a &= \Lambda^a \, _{\mu} \gamma^{\mu} \\ &= (\delta^a \, _{\mu} + \omega^a \, _{\mu})\gamma^{\mu} \\ &= \gamma^a + \omega^a \, _{\mu} \gamma^{\mu} \\ &= \gamma^a + \omega_{b \mu} \eta^{a b} \gamma^{\mu} \\ &= \gamma^a + \omega_{b \mu} \eta^{a [b} \gamma^{\mu]} \\ &= \gamma^a + \frac{1}{2} \omega_{b \mu} (\eta^{a b} \gamma^{\mu} - \eta^{a \mu} \gamma^b) \\ &= \gamma^a + \frac{1}{2} \omega_{\nu} (\eta^{a \mu} \gamma^{\nu} - \eta^{a \nu} \gamma^{\mu}) \\ &= S^{-1} \gamma^a S \\ &= (I - \frac{i}{2} \omega_{\mu \nu} \Sigma^{\mu \nu}) \gamma^a (I + \frac{i}{2} \omega_{\mu \nu} \Sigma^{\mu \nu}) \\ &= \gamma^a - \frac{i}{2} \omega_{\mu \nu} [\gamma^a, \Sigma^{\mu \nu}] \end{align} we have the relation (which can be interpreted as saying that $\gamma^a$ transforms as a vector under spinor representations of Lorentz transformations, as e.g. in Tong's QFT notes) \begin{align} i (\eta^{a \mu} \gamma^{\nu} - \eta^{a \nu} \gamma^{\mu}) = [\gamma^a, \Sigma^{\mu \nu}] \end{align} and we know $\Sigma^{\mu \nu}$, since it is anti-symmetric, must involve a product's of $\gamma$ matrices (because of the 16-dimensional basis formed from Clifford algebra elements), only two by the left-hand side, and from \begin{align} \gamma^{\mu} \gamma^{\nu} &= - \gamma^{\nu} \gamma^{\mu} , \ \ \ \mu \neq \nu, \\ \gamma^{\mu} \gamma^{\mu} &= \gamma^{\nu} \gamma^{\mu} , \ \ \ \mu = \nu, \end{align} we expect that \begin{align} \Sigma^{\mu \nu} &= c [\gamma^{\mu},\gamma^{\nu}] \\ &= c (\gamma^{\mu} \gamma^{\nu} - \gamma^{\nu} \gamma^{\mu}) \\ &= 2 c ( \gamma^{\mu} \gamma^{\nu} - \eta^{\mu \nu}) \end{align} for some $c$ which we constrain by the (vector) relation above \begin{align} i (\eta^{a \mu} \gamma^{\nu} - \eta^{a \nu} \gamma^{\mu}) &= [\gamma^a, \Sigma^{\mu \nu}] \\ &= c [\gamma^a, 2( \gamma^{\mu} \gamma^{\nu} - \eta^{\mu \nu})] \\ &= 2 c [\gamma^a, \gamma^{\mu} \gamma^{\nu}] \\ &= 2 c ( \gamma^{\mu} [\gamma^a,\gamma^{\nu}] + [\gamma^a, \gamma^{\mu}] \gamma^{\nu}) \\ &= 2 c [ \gamma^{\mu} 2( \gamma^a \gamma^{\nu} - \eta^{a \nu}) + 2( \gamma^a \gamma^{\mu} - \eta^{a \mu}) \gamma^{\nu}] \\ &= 4 c [ \gamma^{\mu} ( \gamma^a \gamma^{\nu} - \eta^{a \nu}) + ( \gamma^{\mu} \gamma^{a} + 2 \eta^{a \mu} - \eta^{a \mu}) \gamma^{\nu}] \\ &= 4 c (\eta^{a \mu} \gamma^{\nu} - \eta^{a \nu} \gamma^{\mu}). \end{align} This gives the result $c = i/4$. The generator of Lorentz transformations of gamma matrices is \begin{align} \Sigma^{\mu \nu} &= \dfrac{i}{4} [\gamma^{\mu},\gamma^{\nu}] \\ &= \dfrac{i}{2}(\gamma^{\mu} \gamma^{\nu} - \eta^{\mu \nu}) \ \ \text{i.e.} \\ S &= I - \frac{i}{2} \omega_{\mu \nu} (\frac{i}{4} [\gamma^{\mu},\gamma^{\nu}]) \\ &= I + \dfrac{1}{8} \omega_{\mu \nu} [\gamma^{\mu},\gamma^{\nu}]. \end{align} Using the fact that the gamma matrices transform as a vector under the spinor representation of an infinitesimal Lorentz transformation, \begin{align} [\Sigma^{\mu \nu}, \gamma^{\rho}] = i (\gamma^{\mu} \eta^{\nu \rho} - \gamma^{\nu} \eta^{\mu \rho}) \end{align} we can show the spinor representation of a Lorentz transformation satisfies the Lorentz algebra commutation relations, since for $\rho \neq \sigma$ \begin{align} [\Sigma^{\mu \nu},\Sigma^{\rho \sigma}] &= \frac{i}{2}[\Sigma^{\mu \nu},\gamma^{\rho} \gamma^{\sigma}] \\ &= \frac{i}{2}([\Sigma^{\mu \nu},\gamma^{\rho} ] \gamma^{\sigma} + \gamma^{\rho} [\Sigma^{\mu \nu}, \gamma^{\sigma}]) \\ &= \frac{i}{2}\{ i (\gamma^{\mu} \eta^{\nu \rho} - \gamma^{\nu} \eta^{\mu \rho}) \gamma^{\sigma} + \gamma^{\rho} i (\gamma^{\mu} \eta^{\nu \sigma} - \gamma^{\nu} \eta^{\mu \sigma}) \} \\ &= - \frac{1}{2}\{ \gamma^{\mu} \eta^{\nu \rho} \gamma^{\sigma} - \gamma^{\nu} \eta^{\mu \rho} \gamma^{\sigma} + \gamma^{\rho} \gamma^{\mu} \eta^{\nu \sigma} - \gamma^{\rho} \gamma^{\nu} \eta^{\mu \sigma} \} \\ &= \frac{i}{2}\{ \eta^{\nu \rho} (2 \Sigma^{\mu \sigma} + \eta^{\mu \sigma}) - \eta^{\mu \rho} (2 \Sigma^{\nu \sigma} - \eta^{\nu \sigma}) + (2 \Sigma^{\rho \mu} - \eta^{\rho \mu}) \eta^{\nu \sigma} - (2 \Sigma^{\rho \nu}) - \eta^{\rho \nu}) \eta^{\mu \sigma} \} \\ &= i ( \eta^{\nu \rho} \Sigma^{\mu \sigma} - \eta^{\mu \rho} \Sigma^{\nu \sigma} + \Sigma^{\rho \mu} \eta^{\nu \sigma} - \Sigma^{\rho \nu} \eta^{\mu \sigma} ). \end{align} This method generalizes from $SO(3,1)$ to $SO(N)$, see e.g. Kaku QFT Sec. 2.6, and the underlying reason for doing any of this in the first place is that one seeks to find projective representations which arise due to the non-simple-connectedness of these orthogonal groups. Regarding your question about arbitrary metrics $g_{\mu \nu}$, this method applies to, and arises due to the non-simple-connectedness of, special orthogonal groups, you can't generalize to arbitrary metrics, this is a problem which can be circumvented in supergravity and superstring theory using veilbein's.

References:

  1. Bjorken, J.D. and Drell, S.D., 1964. Relativistic quantum mechanics; Ch. 2.
  2. Kaku, M., 1993. Quantum field theory: a modern introduction. Oxford Univ. Press; Sec. 2.6.
  3. Tong, Quantum Field Theory Notes http://www.damtp.cam.ac.uk/user/tong/qft.html.
  4. http://www.damtp.cam.ac.uk/user/examples/D18S.pdf
  5. Does $GL(N,\mathbb{R})$ own spinor representation? Which group is its covering group? (Kaku's QFT textbook)

I want to know if we take any arbitrary metric $g_{\mu\nu}$ on some space $V$, will the generators defined as $S^{\mu\nu}$ generate a Lie group whose elements are transformations on $V$ that conserve the inner product corresponding to the metric?

Yes. The result is called the "Spin" group. A good overview is in this paper.

In general, Clifford algebras are created from an arbitrary vector space $V$ (over a field $\mathbb{F}$) and a quadratic norm $Q : V \to \mathbb{F}$, where $\mathbb{F}$ is usually (certainly by physicists) taken to be either $\mathbb{R}$ or $\mathbb{C}$. If you have a metric, that's a slightly stronger statement than just having a quadratic norm, so you can certainly use it to construct the Clifford algebra — by defining the norm as $Q(v) = g_{\mu \nu} v^\mu v^\nu$. On the other hand, if you have the norm, you can use it to define the inner product between any two vectors $v$ and $w$ by polarization: $g(v, w) = \frac{1}{2}[Q(v+w) - Q(v) - Q(w)]$. Of course, that only works if you can divide by $2$, which isn't the case for all fields. On the other hand, I can't remember seeing any useful application of Clifford algebra using a field other than $\mathbb{R}$ or $\mathbb{C}$.

In the case of spacetime, the vector space is just the set of $\gamma^\mu$ vectors, which shouldn't be thought of as complex matrices, but rather as just the usual basis vectors: $\hat{t}, \hat{x}, \hat{y}, \hat{z}$. This approach is usually called geometric algebra. The field should actually be taken to be $\mathbb{R}$ (because the complex structure we usually use in quantum mechanics actually shows up automatically in the Clifford algebra). What you get is called the spacetime algebra.

This same logic can be extended to other spaces, of any dimension and signature (including indefinite and degenerate signatures). Any two vectors in the Clifford algebra can be multiplied by each other, and thus the anticommutative product constructed — the result is called a bivector. The set of all bivectors forms the $\mathfrak{spin}$ algebra, where the product is not just the Clifford product but its commutator. More generally, we can take any even number of vectors and take their product. The invertible elements of this form give us the Spin group, related to the bivectors through exponentiation (much as the Lie group is related to the Lie algebra). And they transform vectors by conjugation, which naturally leaves the inner product invariant. So that's the answer to your question.

We also have a sort of inverse to the above:

Every Lie algebra can be represented as a bivector algebra; hence every Lie group can be represented as a spin group.

This result is found here. While they do use a sort of "doubled" Clifford algebra in general, this isn't always necessary. That paper gives a good overview of these issues (as does the one about Spin groups, though not in as much detail).