Is there an elegant way to take a derivative of this linear algebra problem?
$\DeclareMathOperator{\diag}{diag}$ In what follows $d, x, y$ and $\delta$ denote column vectors.
The function $y(d) = d^T P^T P \delta$ can be rewritten as the dot product $\langle Pd, P\delta \rangle$. This is the composition of three functions:
- $y_1 (d) = \begin{bmatrix}d\\ \delta\end{bmatrix}$ which has derivative $Dy_1(d) = \begin{bmatrix} I \\ -2\diag(d) \end{bmatrix}$, where $\diag(d)$ is the diagonal matrix with $(d_1, d_2, d_3)$ along the diagonal and $I$ is the $3 \times 3$ identity matrix.
- $y_2(x, y) = \begin{bmatrix}Px \\ Py \end{bmatrix}$ which has the derivative $Dy_2(x, y) = \begin{bmatrix} P & 0 \\ 0 & P \end{bmatrix}$.
- $y_3 (x, y) = \langle x, y \rangle$ which has the derivative $Dy_3(x, y) = (y^T, x^T)$.
Putting the three things together using the chain rule gives
$$ \begin{align*} Dy(d) &= Dy_3(y_2 \circ y_1(d)) Dy_2(y_1(d)) Dy_1(d) \\ &= Dy_3(Pd, P \delta) \begin{bmatrix} P & 0 \\ 0 & P \end{bmatrix} \begin{bmatrix} I \\ -2\diag(d) \end{bmatrix} \\ &= ((Pd)^T, (P\delta)^T) \begin{bmatrix} P & 0 \\ 0 & P \end{bmatrix} \begin{bmatrix} I \\ -2\diag(d) \end{bmatrix} \\ &= d^TP^TP - 2 \delta^T P^T P \diag(d). \end{align*} $$