Factorise polynomial with real and complex roots

Note that $x^4-4=(x^2+2)(x^2-2)$. This suggests that you might try to express your polynomial as$$(x^2+ax+2)(x^2+bx-2)=x^4+(a+b)x^3+abx^2+2(-a+b)x-4.$$It is now easy to see that all it takes is to choose $a=1$ and $b=4$.


Since there are no rational roots (a rational root should be a divisor of $4$), your attempt of factorization as $$x^4 +5x^3+4x^2+6x-4=(x^2+ax+b)(x^2+cx+d)$$ is a good idea.

Hint. By expanding the right-hand side and by comparing it with the the left-hand side, we have that $bd=-4$. Assuming that $b$ and $d$ are integers, by symmetry, you can try the following couples: $$(b,d)\in\{(4,-1), (2,-2), (-4,1)\}$$ and then solve the corresponding systems with respect to the remaining coefficients $c$ and $d$.

P.S. If you are lucky you will start with the couple $(2,-2)$. Be careful though, there are polynomials, which are "similar" to the given one, such that the other couples work:

i) $(4,-1)$ for $x^4+2x^2-5x-4=(x^2+x+4)(x^2-x-1)$.

ii) $(-4,1)$ for $x^4-4x^2+5x-4=(x^2+x-4)(x^2-x+1)$.


I like the following way.

For all real value of $k$ we have:

$$x^4+5x^3+4x^2+6x-4=\left(x^2+\frac{5}{2}x+k\right)^2-\left(\left(2k+\frac{9}{4}\right)x^2+(10k-6)x+k^2+4\right).$$ Now, we'll choose a value of $k$, for which $2k+\frac{9}{4}>0$ and $$(5k-3)^2-\left(2k+\frac{9}{4}\right)(k^2+4)=0.$$ Easy to see that $k=0$ is valid.

Thus,$$x^4+5x^3+4x^2+6x-4=\left(x^2+\frac{5}{2}x\right)^2-\left(\frac{9}{4}x^2-6x+4\right)=$$ $$=\left(x^2+\frac{5}{2}x\right)^2-\left(\frac{3}{2}x-2\right)^2=(x^2+x+2)(x^2+4x-2).$$ Can you end it now?