Computing $\int_{-π/2}^{π/2} \frac{28\cos^2(θ)+10\cos(θ)\sin(θ)-28\sin^2(θ)}{2\cos^4(θ)+3\cos^2(θ)\sin^2(θ)+m\sin^4(θ)}\ dθ$


Define the function $\mathcal{I}:\mathbb{R}_{>0}\rightarrow\mathbb{R}$ via the trigonometric integral

$$\begin{align} \mathcal{I}{\left(\mu\right)} &:=\int_{-\frac{\pi}{2}}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}+10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}.\\ \end{align}$$


Let $\mu\in\mathbb{R}_{>0}$. Since the integral $\mathcal{I}$ has a symmetric interval of integration, the integral of the odd component of the integrand vanishes identically:

$$\begin{align} \mathcal{I}{\left(\mu\right)} &=\int_{-\frac{\pi}{2}}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}+10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=\int_{-\frac{\pi}{2}}^{0}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}+10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &~~~~~+\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}+10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}-10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}};~~~\small{\left[\theta\mapsto-\theta\right]}\\ &~~~~~+\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{28\cos^{2}{\left(\theta\right)}+10\cos{\left(\theta\right)}\sin{\left(\theta\right)}-28\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{56\cos^{2}{\left(\theta\right)}-56\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=56\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{\cos^{2}{\left(\theta\right)}-\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}.\\ \end{align}$$


Using the double-angle formulas for sine and cosine,

$$\begin{align} \sin{\left(2\theta\right)} &=2\sin{\left(\theta\right)}\cos{\left(\theta\right)},\\ \cos{\left(2\theta\right)} &=\cos^{2}{\left(\theta\right)}-\sin^{2}{\left(\theta\right)}\\ &=2\cos^{2}{\left(\theta\right)}-1\\ &=1-2\sin^{2}{\left(\theta\right)},\\ \end{align}$$

we can rewrite the integral as

$$\begin{align} \mathcal{I}{\left(\mu\right)} &=56\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{\cos^{2}{\left(\theta\right)}-\sin^{2}{\left(\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=56\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{\cos{\left(2\theta\right)}}{2\cos^{4}{\left(\theta\right)}+3\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+\mu\sin^{4}{\left(\theta\right)}}\\ &=56\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{4\cos{\left(2\theta\right)}}{8\cos^{4}{\left(\theta\right)}+12\cos^{2}{\left(\theta\right)}\sin^{2}{\left(\theta\right)}+4\mu\sin^{4}{\left(\theta\right)}}\\ &=56\int_{0}^{\frac{\pi}{2}}\mathrm{d}\theta\,\frac{4\cos{\left(2\theta\right)}}{2\left[1+\cos{\left(2\theta\right)}\right]^{2}+3\sin^{2}{\left(2\theta\right)}+\mu\left[1-\cos{\left(2\theta\right)}\right]^{2}}\\ &=56\int_{0}^{\pi}\mathrm{d}\theta\,\frac{2\cos{\left(\theta\right)}}{2\left[1+\cos{\left(\theta\right)}\right]^{2}+3\sin^{2}{\left(\theta\right)}+\mu\left[1-\cos{\left(\theta\right)}\right]^{2}};~~~\small{\left[\theta\mapsto\frac12\theta\right]}\\ &=112\int_{0}^{\pi}\mathrm{d}\theta\,\frac{\cos{\left(\theta\right)}}{2\left[1+\cos{\left(\theta\right)}\right]^{2}+3\sin^{2}{\left(\theta\right)}+\mu\left[1-\cos{\left(\theta\right)}\right]^{2}}.\\ \end{align}$$

Using the tangent half-angle substitution, the trigonometric integral transforms as

$$\begin{align} \mathcal{I}{\left(\mu\right)} &=112\int_{0}^{\pi}\mathrm{d}\theta\,\frac{\cos{\left(\theta\right)}}{2\left[1+\cos{\left(\theta\right)}\right]^{2}+3\sin^{2}{\left(\theta\right)}+\mu\left[1-\cos{\left(\theta\right)}\right]^{2}}\\ &=112\int_{0}^{\infty}\mathrm{d}t\,\frac{2}{1+t^{2}}\cdot\frac{\left(\frac{1-t^{2}}{1+t^{2}}\right)}{2\left(1+\frac{1-t^{2}}{1+t^{2}}\right)^{2}+3\left(\frac{2t}{1+t^{2}}\right)^{2}+\mu\left(1-\frac{1-t^{2}}{1+t^{2}}\right)^{2}};~~~\small{\left[\theta=2\arctan{\left(t\right)}\right]}\\ &=\int_{0}^{\infty}\mathrm{d}t\,\frac{56\left(1-t^{2}\right)}{2+3t^{2}+\mu\,t^{4}}.\\ \end{align}$$

Setting $\sqrt{\frac{2}{\mu}}=:a\in\mathbb{R}_{>0}$ and $\frac34a=:b\in\mathbb{R}_{>0}$, we then have

$$\begin{align} \mathcal{I}{\left(\mu\right)} &=\mathcal{I}{\left(\frac{2}{a^{2}}\right)}\\ &=\int_{0}^{\infty}\mathrm{d}t\,\frac{56\left(1-t^{2}\right)}{2+3t^{2}+2a^{-2}t^{4}}\\ &=\sqrt{a}\int_{0}^{\infty}\mathrm{d}u\,\frac{56\left(1-au^{2}\right)}{2+3au^{2}+2u^{4}};~~~\small{\left[t=u\sqrt{a}\right]}\\ &=\frac13\sqrt{a}\int_{0}^{\infty}\mathrm{d}u\,\frac{28\left(3-4bu^{2}\right)}{1+2bu^{2}+u^{4}}\\ &=\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{\infty}\mathrm{d}u\,\frac{\left(3-4bu^{2}\right)}{1+2bu^{2}+u^{4}}\\ &=\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{\left(3-4bu^{2}\right)}{1+2bu^{2}+u^{4}}\\ &~~~~~+\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{1}^{\infty}\mathrm{d}u\,\frac{\left(3-4bu^{2}\right)}{1+2bu^{2}+u^{4}}\\ &=\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{\left(3-4bu^{2}\right)}{1+2bu^{2}+u^{4}}\\ &~~~~~+\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{\left(3u^{2}-4b\right)}{1+2bu^{2}+u^{4}};~~~\small{\left[u\mapsto\frac{1}{u}\right]}\\ &=\frac{28}{3}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{\left(3-4b\right)\left(1+u^{2}\right)}{1+2bu^{2}+u^{4}}\\ &=28\left(1-\frac43b\right)\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{1+u^{2}}{1+2bu^{2}+u^{4}}\\ &=28\left(1-a\right)\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{1+u^{2}}{1+2bu^{2}+u^{4}}\\ &=28\left(1-\sqrt{\frac{2}{\mu}}\right)\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{1+u^{2}}{1+2bu^{2}+u^{4}}\\ &=\frac{28\left(\mu-2\right)}{\left(\sqrt{\mu}+\sqrt{2}\right)\sqrt{\mu}}\sqrt[4]{\frac{2}{\mu}}\int_{0}^{1}\mathrm{d}u\,\frac{1+u^{2}}{1+2bu^{2}+u^{4}}.\\ \end{align}$$

Since the integral factor in the last line above is a positive quantity, we don't need to actually solve the integral to determine the sign of $\mathcal{I}$. We have

$$\operatorname{sgn}{\left(\mathcal{I}{\left(\mu\right)}\right)}=\operatorname{sgn}{\left(\mu-2\right)},$$

as you originally conjectured.



This problem is "nice" in the sense that the integrand is really trig function of $2\theta$ $$ I=\int_{-\pi/2}^{\pi/2}\frac{28\cos 2\theta+5\sin2\theta}{\frac12(1+\cos2\theta)^2+\frac34\sin^2 2\theta+\frac{m}4(\cos2\theta-1)^2}\,\mathrm{d}\theta $$ So $$ I=\frac12\int_{-\pi}^\pi\frac{28\cos\phi+5\sin\phi}{\frac12(1+\cos\phi)^2+\frac34\sin^2 \phi+\frac{m}4(\cos\phi-1)^2}\,\mathrm{d}\phi $$ which we can rewrite as a contour integral of a rational function over the unit circle $z=e^{i\phi}$, $-\pi\leq\phi\leq\pi$ in $\mathbb{C}$, hence it is just a matter of computing residues.

So $$ I=\frac12\int_{\mathbb{T}}\frac{28\cdot\frac12(z+z^{-1})+5\cdot\frac1{2i}(z-z^{-1})}{\frac12(1+\frac12(z+z^{-1}))^2+\frac34(-\frac14(z-z^{-1})^2)+\frac{m}4(\frac12(z+z^{-1})-1)^2}\,\frac{\mathrm{d}z}{iz} $$ which simplifies to $$ I=\frac1{2i}\int_{\mathbb{T}} \frac{8 ((28 + 5 i) + (28 - 5 i) z^2)\,\mathrm{d}z}{(m-1)(z^4+1)-4(m-2)(z^3+z)+6(m-3)z^2} $$ The poles are at, if $m\neq 1$, $$ z+z^{-1}=\frac{2(m-2\pm\sqrt{m})}{m-1}. $$ so, since $m>0$ $$\label{eq:poles} z= \begin{cases} 0,\frac12(3\pm\sqrt5)& m=1\\ \frac{2\pm\sqrt{m}\pm\sqrt{3\pm 2\sqrt{m}}}{1\pm\sqrt{m}}&m\neq 1. \end{cases}\tag{$\star$} $$ So all poles are at worst simple unless $m=\frac49$ (in which case we get a double pole at $z=4$ which we can ignore), and $$ I=\pi\sum_{\substack{\lvert z_i\rvert<1\\z_i\in\eqref{eq:poles}}}\operatorname{res}_{z_i}(\dots)+(\text{correction if }\lvert z_i\rvert=1). $$


note that since the function part of the function is odd i.e: $$f(x)=\frac{28\cos^2x+10\cos x\sin x-28\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}$$ $$f(-x)=\frac{28\cos^2x-10\cos x\sin x-28\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}$$ you could notice that the integral can be simplified to: $$\int_{-\pi/2}^{\pi/2}\frac{28\cos^2x+10\cos x\sin x-28\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}dx$$ $$=\int_{-\pi/2}^{\pi/2}\frac{28\cos^2x-28\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}dx$$ $$=2\int_0^{\pi/2}\frac{28\cos^2x-28\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}dx$$ $$=56\int_0^{\pi/2}\frac{\cos^2x-\sin^2x}{2\cos^4x+3\cos^2x\sin^2x+m\sin^4x}dx$$


One route you could try to take is Tangent half-angle substitution, which yields: $$112\int_0^1(1+t^2)\frac{(1-t^2)^2-(2t)^2}{2(1-t^2)^4+3(1-t^2)^2(2t)^2+m(2t)^4}dt$$ the bottom of this fraction can be expanded to: $$2t^8+4t^6+16t^4m-12t^4+4t^2+2$$ this may be factorisable for certain values of $m$