magnetic field outside a solenoid
I tried to address this question in a short article that recently got published in the European Journal of Physics. https://arxiv.org/abs/1610.07876
You can also look at this article by Farley, Price, which uses a nice argument and physically motivates the problem: http://scitation.aip.org/content/aapt/journal/ajp/69/7/10.1119/1.1362694
Also, there is an exact calculation done in this old article: http://paginas.fe.up.pt/~ee08173/wp-content/uploads/2014/03/finite-solenoid.pdf
You first should understand that magnetic field are the circles (of magnetic force) around the wire with current. The direction of circles (counterwise or clockwise) depends on the direction of the currend (forth or back). Now, you can guess what happens to the magnetic field when you curve the wire into a circle. Then, you may also stack cricles of current next to each other to form a cylinder. Now, try to imagine what will be the magnetic field in each of these cases.
The magnetic fields from different currents add up (total field in every point is the sum of fields from different currents at that point).
I agree that symmetry, which tells that cancel is complete, is not obvious here. Because outside the loop, one wire is closer is to the outer region than the wire from the opposite side, which cancels out the first. But, this should give a qualitative picture, what is happening. The complexity of symmetry might be the same as zero gravity inside a massive sphere.
edit I am coming to conclusion that it is impossible to cancel out the outer field completely just because the field approaches infinity as we come closer to the wire from outside and there is nothing close to infinity from the opposite piece of the loop that can cancel this out. Solenoid just cancels out componets of the field perpendicular to its axis and parallel to the plain of the windings since windings are very close and, though current in them flows in one direction, say forward, upper wire induces left magnetic field between the wires whereas bottom wire produces the same right field. So, all the winds that blow in the planes between windings are eliminated. The axial fields inside the solenoid are added together. Axial outer are cancelled to some extent but not completely. But, this is not bad because, though they create the axial field, opposite to what solenodi should produce from its internal part, the irregularities at the edge of solinoid wrap it around and eliminate the negative effect.
See that though the major field is produced from left to right, some field blows from right from right to left outside. Yet, this does not harm the major flow because edges isolate the negative flow.
What goes through the middle must go out one end and back in the other. The magnetic field on the outside is free to spread out and in doing so the Wb/m^2 goes low, but is not zero. You must distinguish between assumptions to prove a theory and real life examples. An infinitely long solenoid (for the theory) would have infinite cross-sectional area outside the solenoid (for the flux return path) so the Wb/m^2 would reduce to zero in math terms. Magnetic fields lines from each coil of solenoid will add, there's no cancelling going on here, or anywhere (just the math with infinity term as denominator means whole term becomes zero.)