Why is the (free) neutron lifetime so long?
NB: I feel like this is a pretty half-assed job, and I apologize for that but having opened my mouth in the comments I guess I have to write something to back it up.
We start with Fermi's golden rule for all transitions. The probability of the transition is $$ P_{i\to f} = \frac{2\pi}{\hbar} \left|M_{i,f}\right|^2 \rho $$ where $\rho$ is the density of final states which is proportional to $p^2$ for massive particles. To find the rate1 for all possible final states we sum over these probabilities incoherently. When the mass difference between the initial and final states is much less than the $W$ mass the matrix element $M_{i,f}$ depends only weakly (hah!) on the particular state and the sum is well approximated by a sum only over the density of states: $$P_\text{decay} \approx \frac{2\pi}{\hbar} \left|M_{}\right|^2 \int_\text{all outcomes} \rho .$$ This sum is collectively called the phase space available to the decay. In these cases the matrix element is also quite small for the reason that Dr BDO discusses.
The phase space computation can be quite complicated as it must be taken over all unconstrained momenta of the products. For decays to two body states it turns out to be easy, there is no freedom in the final states except the $4\pi$ angular distribution in the decay frame (their are eight degrees of freedom in two 4-vectors, but 2 masses and the conservation of four momentum account for all of them except the azimuthal and polar angles of one of the particles).
The decays that you have asked about are to three body states. That gives us twelve degrees of freedom less three constraints from masses, four from conservation of 4-momentum which leaves five. Three of these are the Euler angles describing the orientation of the decay (and a factor of $8\pi^2$ to $\rho$), so our sum is over two non-trival momenta. The integral looks something like $$ \begin{array}\\ \rho \propto \int p_1^2 \mathrm{d}p_1 \int p_2^2 \mathrm{d}p_2 \int \mathrm{d}(\cos\theta) &\delta(m_0 - E_1 - E_2-E_3 ) \\ &\delta(E_1^2 - m_1^2 - p_1^2) \\ &\delta(E_2^2 - m_2^2 - p_2^2) \\ &\delta(E_2^2 - m_2^2 - p_2^2) \\ &\delta(\vec{p}_1 + \vec{p}_2 + \vec{p}_3) \end{array} $$ which is easier to compute in Monte Carlo than by hand. (BTW--the reason for introducing the seemingly redundant integral over the angle $\theta$ between the momenta of particles 1 and 2 will become evident in a little while).
For beta decays the remnant nucleus is very heavy compared to the released energy, which simplifies the above in one limit.
In the case of muon decay, it is not unreasonable to treat all the products as ultra-relativistic, and the above reduces to $$ \begin{array}\\ \rho \propto \int p_1^2 \mathrm{d}p_1 \int p_2^2 \mathrm{d}p_2 \int \mathrm{d}(\cos\theta) &\delta(m_0 - E_1 - E_2 - E_3 ) \\ &\delta(E_1 - p_1) \\ &\delta(E_2 - p_2) \\ &\delta(E_3 - p_3) \\ &\delta(\vec{p}_1 + \vec{p}_2 + \vec{p}_3) \\ = \int p_1^2 \mathrm{d}p_1 \int p_2^2 \mathrm{d}p_2 \int \mathrm{d}(\cos\theta) &\delta(m_0 - p_1 - p_2 - p_3 ) \\ &\delta(\vec{p}_1 + \vec{p}_2 + \vec{p}_3) \\ = \int p_1^2 \mathrm{d}p_1 \int p_2^2 \mathrm{d}p_2 \int \mathrm{d}(\cos\theta) &\delta(m_0 - p_1 - p_2 - \left|\vec{p}_1 + \vec{p}_2\right| )\\ = \int p_1^2 \mathrm{d}p_1 \int p_2^2 \mathrm{d}p_2 \int \mathrm{d}(\cos\theta) &\delta\left(m_0 - p_1 - p_2 - \sqrt{p_1^2 + p_2^2 - p_1p_2\cos\theta} \right) \end{array} $$ The integral over the angle will evaluate to one in some regions and zero in others and as such is equivalent to correctly assigning the limits of the other two integrals, so writing $\delta m = m_0 - m_1 - m_2 - m_3$ we get $$ \begin{array} \rho & \propto \int_0^{\delta m/2} p_1^2 \mathrm{d}p_1 \int_0^{\delta m-p_1} p_2^2 \mathrm{d}p_2 \\ & \propto \int_0^{\delta m/2} p_1^2 \mathrm{d}p_1 \left[ \frac{p_2^3}{3}\right]_{p_2=0}^{\delta m-p_1} \\ & \propto \int_0^{\delta m/2} p_1^2 \mathrm{d}p_1 \frac{(\delta m - p_1)^3}{3} \end{array} $$ which I am not going to bother finishing but shows that that phase space can vary as a high power of the mass difference (up to the sixth power in this case).
1 The lifetime of the state is inversely proportional to the probability
You can estimate the neutron lifetime using dimensional analysis. Beta decay is correctly described by the well known four-fermion Fermi theory, so the amplitude must be proportional to the coupling $G_F\approx10^{-5}\text{GeV}^{-2}$ (the Fermi constant). The decay rate is proportional to the squared amplitude:
$$\Gamma\propto G_F^2\thinspace.$$
$\Gamma$ has units of Mass while $G_F^2$ has units of $[\text{Mass}]^{-4}$, so to get the units wright we must have
$$\Gamma\propto G_F^2 \Delta^5$$
where $\Delta$ is some quantity that has units of mass. The relevant mass scale in neutron decay is the mass difference between neutron and proton, so $\Delta=m_n-m_p\approx10^{-3}\text{GeV}$.
To be a bitsy more accurate one can try guess the $\pi$ dependence of the decay rate. This comes from the phase space of a 3-body decay, which usually goes as
$$(2\pi)^4\times\Big[\thinspace(2\pi)^{-3}\times(2\pi)^{-3}\times(2\pi)^{-3}\Big]\times (\pi^2)\ \propto\ \pi^{-3}\thinspace.$$
The first factor comes from the four momenta conservation delta function, the three $(2\pi)^{-3}$ in the bracket come from the integration measure of the 4-momenta of each outgoing particle and the last factor comes from integrating the angle variables. One finally gets the following estimation
$$\Gamma\propto \frac{1}{\pi^3}G_F^2(m_n-m_p)^5$$
If one plugs in all the numbers, the estimated lifetime of the neutron reads
$$\tau_{\text{neutron}}\ =\ \Gamma^{-1}\ \approx\ \pi^3 \ \text{sec.}$$
This is a bit shorter than the real value which is at least one order of magnitude larger. But it explains why the neutron's lifetime is so large (inverse of the 5th power of the small mass difference) with respect to other weak decay processes.
As you correctly state, the neutron decay is an decay due to the weak interaction, these are quite a bit slower than other decays due the mass of the intermediate W boson, 81GeV, which slows the reaction, additionally the neutron decay only liberates a small amount of energy, around 1 MeV, it is the ratio of the liberated energy to the mass of the W which sets the speed of the reaction which is thus much slower than other decays as all the other particle decay release much more energy.