When is $(a^2+b)(b^2+a)$ a power of $2$?

The product $(a^2+b)(a+b^2)$ is a power of two if and only if both factors are: $$\begin{array}{rcl} a^2+b&=&2^r\\ a+b^2&=&2^s \end{array}$$

If $a=b$, then the equation $a^2+a=a(a+1)=2^r$ has only one solution, namely, $a=1$. So we can WLOG assume that $a>b$. Thus, $r>s$.

Since $2^s=a^2+b\geq2^2+1$, we have that $s>1$.

Since $s>1$, $a$ and $b$ have the same parity. Substracting the equations and factoring we have: $$(a-b)(a+b-1)=2^s(2^{r-s}-1)$$ Since $a+b-1$ is odd, $2^s$ divides $a-b$, that is, $a=2^sk+b$, $k\geq1$. But $a=2^s-b^2<2^s$, a contradiction.


If $a$ has lower power of $2$ than $b$:

  $a+b^2$ will be an odd multiple of a power of $2$ and hence won't be a power of $2$

Similarly if $b$ has lower power of $2$ than $a$

Therefore $a,b$ must have the same power $p$ of $2$

Let $a = 2^p x$ where $x$ is odd

Let $b = 2^p y$ where $y$ is odd

Then $(a^2+b)(b^2+a) = 2^{2p} ( 2^p x^2 + y ) ( 2^p y^2 + x )$

Thus $2^p x^2$ is odd, and hence $p = 0$, and so $a,b$ are both odd

If $a = b$:

  $(a^2+b)(b^2+a) = a^2 (a+1)^2$

  Thus $a = 1$ otherwise $a^2$ is not a power of $2$

If $a \ne b$:

  $(a-b)(a+b-1) = 2^q z$ where $q$ is positive and $z$ is odd

  $a = 2^q + b$

  $(a^2+b)(b^2+a) = ( 2^{2q} + 2^{q+1} b + b^2 ) ( 2^q + b^2 + b )$

  Thus $2 | b$ otherwise $2^{2q} + 2^{q+1} b + b^2$ will not be a power of $2$

  Contradiction

Therefore $(a,b) = (1,1)$