How do I evaluate the following definite integral: $\int^1_0\frac{x^2\ln x}{\sqrt{1-x^2}}dx$?

Let $x=\sin y$

$$I=\int_0^1\dfrac{x^2\ln x}{\sqrt{1-x^2}}dx=\int_0^{\pi/2}\sin^2y\ln(\sin y)dy$$

$$2I=\int_0^{\pi/2}\ln(\sin y)dy-\int_0^{\pi/2}\cos2y\ln(\sin y)dy$$

For the first half, see How do you evaluate the integral $\int_{0}^{\frac{\pi}{2}} \log(\sin x) dx$?

For the second,

$$\int\cos2y\ln(\sin y)dy=\ln(\sin y)\int\cos2y\ dy-\int\left(\dfrac{d\ \ln(\sin y)}{dy}\int\cos2y\ dy\right)dy=?$$

Hope you can take it from here!

Let $~I(k)=\displaystyle\int_0^1\frac{x^k}{\sqrt{1-x^2}}~dx.~$ Then our integral becomes $I'(2).~$ At the same time, a simple

trigonometric substitution yields $~I(k)=\displaystyle\int_0^\tfrac\pi2\sin^kx~dx.~$ But the latter is nothing more than

the famous Wallis integral, whose relation to the beta function is well-known. We do not need

to go through the former to show a link to the latter. Indeed, a simple substitution of the form

$t=x^2$ would have sufficed to establish such a connection. We have $I(k)=\dfrac\pi{k~B\bigg(\dfrac12~,~\dfrac k2\bigg)}$ .

Differentiating with regard to k, and using the relation between the beta and $\Gamma$ functions, we

will inevitably have to employ digamma functions, which are nothing more than harmonic

numbers in disguise. In particular, Euler's formula for extending the latter to non-natural

arguments will be of great help in evaluating $\psi_0\bigg(\dfrac32\bigg).~$ The final result should be

$$I'(2)~=~\dfrac\pi8\bigg(1-2\ln2\bigg).$$

Rewrite the integral as$$I(a)=\lim\limits_{a\to0}\frac {\partial}{\partial a}\int\limits_0^1dx\,\frac {x^{a+2}}{\sqrt{1-x^2}}$$And make the substitution $u=x^2$. You should get the resulting integral in terms of the beta function.