Orbital speed for a circular orbit?

I think these are two separate questions that should be approached separately.

1) "Why isn't $m$ in in the first equation?" The mass of a body does change the force acting on it. But the mass of a body also changes its acceleration. If you increase the mass of an object it feels a larger force, but it's also harder to move. The equation for gravitational force is $F = \displaystyle GmM/{r^2}$, while the equation for acceleration is $a = F/m$. Glue to two together and you get $a = GM/r^2$.

2) "Why is $m$ in the second equation?" Think about the moon and the earth. The earth is pulling on the moon, but the moon is also pulling on the earth! The two bodies actually orbit around their common center of mass. This is important for the relative velocity: we need to add how fast the earth is orbiting to how fast the moon is orbiting. That's why you have the $m$ term.

The relative velocity in a circular orbit is indeed: $v_\text{rel} = \sqrt{ \dfrac{G(m_1+m_2) }{ r_\text{rel}} }$

The relative velocity is the sum of the barycentric velocity of each body (the velocity of each body with respect to the inertial center of mass): $$ v_\text{rel} = v_1 + v_2 $$

$$ v_1 = \frac{m_2}{m_1 + m_2} v_\text{rel}, \quad \quad v_2 = \frac{m_1}{m_1 + m_2} v_\text{rel} $$

Similarly, $r_\text{rel}$ is the relative separation of the two bodies.

$$ r_\text{rel} = r_1 + r_2 $$

Where $ r_1 $ and $ r_2 $ are the distances of each body from their mutual center of mass

$$ r_1 = \frac{m_2}{m_1 + m_2} r_\text{rel}, \quad\quad r_2 = \frac{m_1}{m_1 + m_2} r_\text{rel}$$

The relative velocity is also given by: $$ v_\text{rel} = \sqrt{\frac{G m_1}{r_2}} = \sqrt{\frac{G m_2}{r_1}} $$

The barycentric velocity of body 2 is then: $$ v_2 = m_1 \sqrt{ \frac{G}{(m_1+m_2) \, r_\text{rel}} } = \frac{m_1}{m_1+m_2} \sqrt{ \frac{G m_1}{r_2} } $$

This is the actual velocity of body 2 in the inertial frame of the barycenter.