How to Normalize a Wave Function?
The proposed "suggestion" should actually be called a requirement: you have to use it as a normalization condition. This is because the wavefunctions are not normalizable: what has to equal 1 is the integral of $|\psi|^2$, not of $\psi$, and $|\psi|^2$ is a constant. Just like a regular plane wave, the integral without $N$ is infinite, so no value of $N$ will make it equal to one.
One option here would be to just give up and not calculate $N$ (or say that it's equal to 1 and forget about it). This is not wrong! The functions $\psi_E$ are not physical - no actual particle can have them as a state. Physical states $\psi(p)$ are superpositions of our basis wavefunctions, built as
$$\psi(p) = \int dE\, f(E) \psi_E(p)$$
with $f(E)$ some function. This new wavefunction is physical, and it must be normalized, and $f(E)$ handles that job - you have to choose it so that the result is normalized.
But there are two reasons we decide to impose $\langle E | E' \rangle = \delta(E-E')$. One is that it's useful to have some convention for our basis, so that latter calculations are easier. Having a delta function is unavoidable, since regardless of the normalization the inner product will be zero for different energies and infinite for equal energies, but we could put some (possibly $E$-dependent) coefficient in front of it - that's just up to convention.
The other reason is that if you dig a little deeper into the normalization of the $\psi(p)$ above, the delta function appears anyway. We have
$$\langle \psi | \psi \rangle = \int dp\, \int dE\, \int dE'\, f(E)^* f(E') \psi_E^*(p) \psi_{E'}(p),$$
and you can see that the inner product $\langle E | E' \rangle$ is right there, in the $E$ integral. So we have to use the fact that it is proportional to $\delta(E-E')$, and it's neater to fix the constant of proportionality beforehand.
So to recap: having $\langle E | E' \rangle \propto \delta(E-E')$ just falls out of the definition of the $\psi_E(p)$, and it's also obviously the manifestation of the fact that stationary states with different energies are orthogonal. We're just free to choose what goes in front of the delta function, which is equivalent to giving a (possibly energy dependent) value for $N$. Using $\delta(E-E')$ by itself is just the simplest choice, but sometimes other factors are used.
Now, actually calculating $N$ given this convention is pretty easy: I won't give you the answer, but notice that when you calculate the inner product of two wavefunctions with different energies (that is, the integral of $\psi_E^* \psi_{E'}$), the parts with $p^3$ in the exponential cancel, because they don't depend on the energy. What's left is a regular complex exponential, and by using the identity
$$\int_{-\infty}^\infty dx\, e^{ikx} = 2\pi \delta(k)$$
(which is rigorous enough for our purposes), you show that the whole thing must be proportional to $\delta(E'-E)$, and derive the value of $N$ from there.