Is this integral $\int_0^\infty\frac{\cos(a x+ 2b \arctan x)}{x^2+1}dx$ exactly zero when $b\in\mathbb{N}$?

This can, as the author suspected, be approached by contour integration.

Let $a>0$ and $n\in\mathbb{Z}$. As Mason pointed out, the integral in question is equal to $$\int_0^{\pi/2}\cos(a\tan(x)+2nx)dx$$ Using that the integrand is even we may extend the bounds to $(-\pi/2,\pi/2)$ then perform the substitution $t=2x$ to find this is $$\frac{1}{4}\int_{-\pi}^\pi \cos(a\tan(t/2)+nt)dt$$ Notice that this is is equal to $$\frac{1}{4}\int_{-\pi}^\pi \exp(ia\tan(t/2)+int)dt$$ since the odd part of $e^{ix}=\cos(x)+i\sin(x)$ cancels. Now by expanding the tangent in terms of exponentials and performing some algebra we find $$\tan(t/2)=\frac{1}{i}\frac{e^{it}-1}{e^{it}+1}$$ Thus we may rewrite our integral as a contour integral with $z=e^{i t}$, $dt=\frac{1}{iz}dz$. Since our bounds are $t\in(-\pi,\pi)$, the contour is the unit circle. Thus our integral is $$\frac{1}{4i}\oint \exp(a\frac{z-1}{z+1})z^{n-1}dz$$ (Notice $e^{int}=z^n$.) By the Residue Theorem, this contour integral is equal to $$\frac{\pi}{2}\text{ Res}_{z=0} \exp(a\frac{z-1}{z+1})z^{n-1}$$ This residue is the coefficient of $z^{-n}$ in $\exp(a\frac{z-1}{z+1})$. Since this function is analytic inside the unit disk, this coefficient is $0$ for $n>0$ (with $n\in\mathbb{Z}$). The value of the integral for negative integral $n$ is given by the $|n|$th Taylor coefficient of $\frac{\pi}{2}\exp(a\frac{z-1}{z+1})$.


Maybe not a full answer, but part of the way:

Let $a=0$. Then we have

$$\int_0^\infty\frac{\cos(0+ 2b \arctan x)}{x^2+1}dx=\left[\frac{\sin\left(2b \arctan(x)\right)}{2b}\right]_0^\infty\\ =\frac{\sin\left(2b\cdot\frac{\pi}{2}\right)}{2b}-\frac{\sin(2b\cdot0)}{2b}\\ = \frac{\sin(b\pi)}{2b}$$

This is obviously zero if and only if b is an integer (except zero).

Currently I am not fully aware why this behavior remains when $a\neq0$, maybe I find a solution for that later on.