Principle of equivalence between a magnetic dipole and a closed loop with electric current
The papers you should read for this question are Vaidman, Lev. "Torque and force on a magnetic dipole." Am. J. Phys 58.10 (1990): 978-983 (Paywall-free version) and Haus, H. A., and P. Penfield. "Force on a current loop." Physics Letters A 26.9 (1968): 412-413. (References 8 and 9 in Vaidman's paper are also worth reading for more context.) The gist of the answer is this: if and only if there are no magnetic monopoles, magnetic dipoles and current loops are equivalent.
As you have identified in your point (1), there is indeed a problem with the term magnetic dipole. Let's unpack the term starting with dipole.
The simplest system in electromagnetism is the electric point charge at rest, which produces a spherically symmetric $1/r$ potential (it's easier to work with the potential here). This is called a monopole field, and a point charge is a monopole.
Now consider two equal but opposite charges a distance $a$ from each other. This breaks rotational symmetry so the field will not be spherically symmetric, i.e., it is angle-dependent. Now expand the potential at a distance $r \gg a$ from the charges in powers of $1/r$. Since electromagnetism is linear, the $1/r$ terms are equal but opposite and cancel, but there remains a term of order $1/r^2$. Because the potential is a scalar, it must have the form $\mathbf p \cdot \mathbf r / r^3$ where $\mathbf p$ is a vector called the dipole moment. You can continue the expansion; the next term is $Q_{ij} x_i x_j / r^5$ where the tensor $Q_{ij}$ is the quadrupole moment, and so on. (The names reflect the minimum number of point charges you need to have a nonzero moment of that order: one for a monopole, two for a dipole, four for a quadrupole... All the details of this multipole expansion and what it's good for are in Jackson, of course.)
So by a dipole field we mean a field whose potential looks like $\mathbf p \cdot \mathbf r /r^3$, and its source is an electric dipole. Of course the magnetic field has a vector potential rather than a scalar potential, so by a magnetic dipole field we should mean one where the vector potential is like $\mathbf A \sim \mathbf m \times \mathbf r / r^3$, and its source is a magnetic dipole.
Now, one way to construct a magnetic dipole is by the obvious analogy: take two magnetic charges, i.e., two magnetic monopoles at a small distance. Well, that's easier said than done because no one has found any magnetic monpoles. We'll have to go with currents, then. Since the $1/r$ expansion is possible only if the size of the system is finite and charge is conserved, we'll have to use current loops, and conversely, any current loop will be a magnetic dipole.
In this sense and this sense only magnetic dipoles and current loops are equivalent. If you were to find some magnetic monopoles and arrange them such that the dipole moment is $\mathbf m$, then the force on the system is $$\mathbf F_\text{MM} = (\mathbf m \cdot \nabla) \mathbf B - \frac{1}{c}\dot{\mathbf m} \times \mathbf E$$ where $\dot{\mathbf m}$ is the time derivative, whereas the force on a current loop is $$\mathbf F_\text{CL} = \nabla (\mathbf m \cdot \mathbf B) - \frac{1}{c}\frac{d}{dt} ( \mathbf m \times \mathbf E ).$$ If you expand $\mathbf F_\text{CL}$ and use Ampere's law with Maxwell's current, you see that these forces differ by $ k\mathbf m \times \mathbf J$ where $\mathbf J$ is the current density and $k$ is a constant that depends on your unit system. Clearly these magnetic dipoles are not equivalent.
(However, several authors erroneously calculate the force using the magnetic charge model and think it must be true for current loops, which, as shown by Vaidman, is not the case.)
There is one objection and that is that we know about spin, the intrinsic magnetic moment of particles such as electrons. I don't think it is obvious whether spin should be treated as a magnetic charge dipole, or as a current loop. For an elementary particle to be a current loop certainly seems strange, but it's not really less strange to think about it as a system of magnetic monopoles. One would think it's an experimental question, then, but Bohr and Pauli argued in the 20s that the spin of an individual electron is rather inaccessible to experiment, see Morrison, Margaret. "Spin: All is not what it seems." Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics 38.3 (2007): 529-557 for an account.) In any experiment with electrons, the Lorentz force would anyway dominate the magnetic dipole force, so one would have to turn to neutrons, which come with other difficulties. Vaidman discusses this briefly.
However, theoretically, if the situation is analyzed correctly, that is, using the the Foldy-Wouthuysen transformation (the orignal paper is Foldy, Leslie L., and Siegfried A. Wouthuysen. "On the Dirac theory of spin 1/2 particles and its non-relativistic limit." Physical Review 78.1 (1950): 29 which is a real gem and should be read by everyone studying quantum mechanics.) it is found that the current-loop model is correct. You can square it with the contradiction between current loop and elementary particle by realizing that this is a quantum mechanics thing, and in quantum mechanics you don't have to have point particles. In fact, you can't, by Heisenberg's principle. The electron is always a bit spread out, and in such a way as to produce a current loop magnetic dipole moment.
Yes it is a magnet, the size is not important - just the usual N and S poles, two poles, "di"poles.
When current runs through a solenoid, a cylindrical coil of wire (that is to all practical purposes closed when connected to a dc power source), a magnetic dipole is created, N at one end of the cylinder S at the other.
You could remove the solenoid and replace it with the right size magnet the magnetic field there would be the same. (they are equivalent)
It's not an analogy, it is an actual equivalence.