Infinitely charged wire and Differential form of Gauss' Law
The infinitely long wire has an infinite charge $$Q~=~\lambda \int_{-\infty}^{\infty} \! dz ~=~ \infty,\tag{1}$$ and EM has an infinite range, so one shouldn't be surprised to learn that the result $$\phi(r)~=~ \frac{\lambda}{4 \pi \epsilon_0} \int_{-\infty}^{\infty} \frac{dz}{\sqrt{z^2+r^2}} ~=~ \frac{\lambda}{4 \pi \epsilon_0} \left[ {\rm arsinh} \left(\frac{z}{r}\right)\right]_{z=-\infty}^{z=\infty} ~=~\infty \tag{2}$$ is infinite. (From a mathematical point of view, the integrand fails to be integrable wrt. the $z$ variable.) A similar situation happens often in Newtonian gravity if the total mass is infinite, see e.g. this question.
However, as Pygmalion mentions in his answer, the electric field $\vec{E}$ is well-defined for $r\neq 0$, and the corresponding integrand is integrable wrt. the $z$ variable. E.g. the radial component (in cylindrical coordinates) reads $$E_r(r)~=~ \frac{\lambda r}{4 \pi \epsilon_0} \int_{-\infty}^{\infty} \frac{dz}{(z^2+r^2)^{3/2}} ~=~ \frac{\lambda r}{4 \pi \epsilon_0} \left[ \frac{z}{r^2\sqrt{z^2+r^2}}\right]_{z=-\infty}^{z=\infty} ~=~\frac{\lambda}{2\pi\epsilon_0 r}\tag{3} $$ for $r\neq 0$.
Alternatively, apply Gauss' law $$d\Phi_{E} ~=~\frac{dQ}{\epsilon_0},\tag{4}$$ using an infinitesimally thin disk perpendicular to the wire. The disk has radius $r$ and thickness $dz$. The total electric flux $d\Phi_{E}$ out of the disk is $$ E_r \cdot 2\pi r dz ~=~ \frac{\lambda dz}{\epsilon_0},\tag{5}$$ which leads to the same electric field $E_r$.
This electric field $\vec{E}=-\vec{\nabla}\phi$ is consistent with a potential of the form $$\phi(r) ~=~-\frac{\lambda}{2\pi\epsilon_0}\ln r \qquad \text{for}\qquad r\neq 0.\tag{6}$$
You're not calculating the potential here, you're calculating the self energy of the wire. The self energy is the energy required to bring charges from infinity to create the wire. And this is obviously infinite.
The potential of a wire is defined as the energy required to bring a point charge from infinity to a point on the wire. Note that, by this definition, the potential of the wite is infinite as well. Also note that not all objects have a defined "potential". A uniformly charged sphete has a varying potential at different points, so we cannot assign a "potential" to it (it has a self energy though).
No, the differential form of Gauss' law cannot be used simce $\rho$ is infinite at points. I guess you could use limits and do it via the differential form. Or you could convert it to the integral form via Stokes law and do it(sort of cheating). Usually its best to use the integral form when you have infinite volume charge densities.