How can we show that $\int_{0}^{2\pi}{x\over \phi-\cos^2(x)}\mathrm dx=2\pi^2?$

Here is a route, I leave it to you to fill in the details: $$ \begin{aligned} \int_0^{2\pi}\frac{x}{\phi-\cos^2x}\,dx &=\int_0^\pi\frac{x}{\phi-\cos^2x}\,dx+\int_0^{\pi}\frac{x+\pi}{\phi-\cos^2(x+\pi)}\,dx\\ &=2\int_0^\pi\frac{x}{\phi+\sin^2x-1}\,dx+\int_0^\pi\frac{\pi}{\phi-\cos^2x}\,dx\\ &=2\frac\pi2\int_0^\pi\frac{1}{\phi+\sin^2x-1}\,dx+\int_0^\pi\frac{\pi}{\phi-\cos^2x}\,dx\\ &=2\pi\int_0^\pi\frac{1}{\phi-\cos^2x}\,dx\\ &=4\pi\int_0^{\pi/2}\frac{1}{\phi-\cos^2x}\,dx\\ &=4\pi\int_0^{\pi/2}\frac{1}{\cos^2x}\frac{1}{\phi\tan^2x+\phi-1}\,dx\\ &=4\pi\biggl[\frac{1}{\sqrt{\phi(\phi-1)}}\arctan\Bigl(\sqrt{\frac{\phi}{\phi-1}}\tan x\Bigr)\biggr]_0^{\pi/2}\\ &=4\pi\frac{1}{\sqrt{\phi(\phi-1)}}\frac\pi2\\ &=2\pi^2. \end{aligned} $$


Since it may be in any case instructive, let me try to give the full residue-method solution.The integration contour $\Gamma$ in the complex plane. Choosing the branch cut of the logarithm along the positive real axis, let us apply the residue theorem to the corresponding keyhole contour $\Gamma$, which, as sketched in the figure, goes around the point $z=z_1=\sqrt{2\phi-3}=-z_2$ from above and below. We obtain: $$ \oint_\Gamma \frac{4z\log z}{z^4-2(2\phi-1)z^2+1}dz = i2\pi\frac{\log z_2}{z_2^2-(2\phi-1)} = -i\pi\log z_1 +\pi^2, $$ where we have used that $z_{2}$ is the only root of the denominator enclosed in the integration contour and we have substituted $\log z_2 = \log(z_1e^{i\pi})=\log z_1+i\pi$.

The integration contour $\Gamma$ can be split into its six pieces: the unit circle $C_1$, two segments above and below the real axis, two half-circles $\gamma^\pm_\varepsilon$ around $z_1$ and a small circle around the origin. As the radius $\varepsilon$ of the small circle and of $\gamma_\varepsilon^\pm$ tends to zero, these pieces yield (care is needed because we are comparing quantities across a branch cut): $$\begin{aligned} \oint_\Gamma \frac{4z\log z}{z^4-2(2\phi-1)z^2+1}dz =&\ \int_0^{2\pi}\frac{t}{\phi-\cos^2t}dt\\ &- \mathrm{PV} \int_0^1 \frac{4x(\log x +i2\pi)}{x^4-2(2\phi-1)x^2+1}dx\\ &-i\pi \frac{\log z_1 + i2\pi}{z_1^2-(2\phi-1)}\\ &+0\\ &+\mathrm{PV} \int_0^1 \frac{4x\log x }{x^4-2(2\phi-1)x^2+1}dx\\ &-i\pi \frac{\log z_1}{z_1^2-(2\phi-1)}\\ =&\int_0^{2\pi}\frac{t}{\phi-\cos^2t}dt -\pi^2\\ &-i8\pi\, \mathrm{PV}\int_0^1\frac{x}{x^4-2(2\phi-1)x^2+1}dx+i\pi\log z_1, \end{aligned} $$ where $\mathrm{PV}$ denotes the Cauchy principal value. By comparison of the real parts, $$ \int_{0}^{2\pi}\frac{t}{\phi-\cos^2t}dt=2\pi^2. $$ Furthermore, by comparison of the imaginary parts, we get the bonus identity $$ \mathrm{PV} \int_{0}^{1}\frac{ x}{x^4-2(2\phi-1)x^2+1}dx=\frac{1}{4}\log z_1=\frac{1}{8}\log(\sqrt{5}-2). $$ Alio modo: let us consider instead the complex function $$ f(z)=\frac{z}{\phi-\cos^2z}=\frac{-4ze^{i2z}}{(e^{i2z}-e^{i2\zeta_1})(e^{i2z}-e^{i2\zeta_3})}, $$ where $\zeta_1=-i\log\sqrt{2\phi-3}=\zeta_2-\pi$ and $\zeta_3=-i\log\sqrt{2\phi+1}=\zeta_4-\pi$. This function has simple poles at $\zeta_1$, $\zeta_2$ in the upper half plane and $\zeta_3$, $\zeta_4$ in the lower half plane, up to integer multiples of $2\pi$. enter image description here Denoting by $\Pi$ the rectangular contour sketched in the figure , with half-circular indents of radius $\varepsilon$ around $\zeta_1$ and $\zeta_1+2\pi$, we have $$ \oint_\Pi f(z) dz=i2\pi \,\text{Res}f(\zeta_2)=i2\pi\lim_{z\to\zeta_2}(z-\zeta_2)f(z)=\pi^2-i\pi\log\sqrt{2\phi-3}. $$ Furthermore, in the limits $M\to\infty$ and $\varepsilon\to0$, from the various pieces of the integration contour we get: $$\begin{aligned} \oint_\Pi f(z) dz =&\ \int_0^{2\pi}\frac{t}{\phi-\cos^2t}dt\\ &\ +\mathrm{PV} \int_0^\infty \frac{2\pi+iy}{\phi-\cosh^2y}idy\\ &\ -0\\ &\ -\mathrm{PV} \int_0^\infty \frac{iy}{\phi-\cosh^2y}idy\\ &\ -i\pi \,\text{Res}f(\zeta_1)\\ &\ -i\pi \,\text{Res}f(\zeta_1+2\pi)\\ =&\ \int_0^{2\pi}\frac{t}{\phi-\cos^2t}dt-\pi^2\\ &\ i2\pi\, \mathrm{PV} \int_0^\infty \frac{dy}{\phi-\cosh^2y}+i\pi\log\sqrt{2\phi-3}. \end{aligned}$$ Again, by comparison: $$ \begin{aligned} \int_0^{2\pi}\frac{t}{\phi-\cos^2t}dt&=2\pi^2\\ \mathrm{PV} \int_0^\infty \frac{dy}{\phi-\cosh^2y}&=\frac{1}{2}\log(2\phi-3). \end{aligned} $$


Instead of starting from your integral, I will start from the integral in the solution by @Mickep, i.e,

$$\tag{1}I=4\pi\int_0^{\pi/2}\frac{1}{\phi-\cos^2x}\,dx=\pi\underbrace{\int_0^{2\pi}\frac{1}{\phi-\cos^2x}\,dx}_{J} $$

on which it will be simpler to explain how to compute an integral using residue calculus than on the original integral. I will not enter into details. For this I refer you to the numerous lecture notes on residue calculus, for example here.

The main idea is to transform this integral into a circuit integral along a closed parametrized arc, which is here, in a natural way, the unit circle $\gamma$, traversed in the direct orientation.

Let $z=e^{ix}$, with $dz=ie^{ix}dx$. Thus, $J$ becomes, using Euler formula $\cos(x)=\frac12(e^{ix}+e^{-ix})$: $$J=\int_{\gamma}\dfrac{-i dz/z}{\Phi-\dfrac{(z+1/z)^2}{4}}$$

Expanding and reducing, we get:

$$\tag{2}J=-i \underbrace{\int_{\gamma}\dfrac{4z dz}{-z^4+2(2\Phi-1)z^2-1}}_{K}$$

The integrand has four poles (roots of the denominator), all of them real:

$$\begin{cases}z_1=\sqrt{2 \Phi -3} \ \ \ \text{and } \ \ \ z_2=-z_1=-\sqrt{2 \Phi -3}\\ z_3=\sqrt{1+2 \Phi} \ \ \ \text{and } \ \ \ z_4=-z_3=-\sqrt{1+2 \Phi}\end{cases}$$

Only $z_1$ and $z_2$ are inside contour $\gamma.$

The residue theorem says that a complex integral $\int_{\gamma}...$ whatever the sufficiently regular closed contour $\gamma$, is equal to the sum of residues at the poles situated inside the contour multiplied by $2i\pi$.

Now, what is the residue of a function of the form $\dfrac{f(z)}{g(z)}$ at a pole $z_0$? It is the number given by the following formula:

$$\dfrac{f(z_0)}{g'(z_0)}.$$ Although this is not the most general definition, this expression covers a large number of cases (see remark below).

Thus, the residue theorem gives:

$$K=2i\pi\left(\dfrac{4z_1 }{-4z_1^3+4(2\Phi-1)z_1}+\dfrac{4z_2}{-4z_2^3+4(2\Phi-1)z_2}\right)$$

$$K=2i\pi\left(\dfrac{1}{-z_1^2+(2\Phi-1)}+\dfrac{1}{-z_2^2+(2\Phi-1)}\right)$$

As $z_1^2=z_2^2=2\Phi-3$, the denominators have a common value, which is $-2$. Thus $K=-2i\pi$. Plugging this value into (2) and then (1) gives the awaited result.

Remark: The definition of residues I have given is only valid for simple poles; if $z_0$ is for example a double root of the denominator, it is understandable that we are in trouble because $g'(z_0)=0$. There exist specific formulas for these cases.