Why do clothes dry at room temperature?

Microscopically, both the water molecules in the air and the water molecules on the clothing are rapidly moving around due to their thermal energy. Every once in a while, a molecule on the clothing will have enough energy to break free; every once in a while, a molecule in the air will stick to your clothes. Because the humidity in your room is less than 100%, the first process will happen more often, so water will go into the air. Conversely, if you put your clothes in a sauna, where the humidity is more than 100%, your clothes will get more wet over time.

Now let's look macroscopically. Why should evaporation happen at all, if it costs energy to do it? The reason is that there's "more room" for the water molecules in the air than on your clothes, so it's more likely for things to bounce into the air (which is big) than land on your clothes (which are small). Formally, we say that the process is entropy-driven. (Since entropy counts available microstates, this is just the same thing.)

Increasing entropy and decreasing energy are separate goals. In this case, the energy and entropy effects oppose each other, but entropy wins; in general, you can tell which one wins by looking at the change in Helmholtz free energy, $F = U - TS$.

The air molecules are like little balls constantly hitting the surface of water. The air molecules strike at different velocities and the one with higher velocities knock out, lets say, 2 water molecules. Other molecules rush to the site. The knocked out water molecule is still bombarded by other air molecules which drift it away.

Now, the density of knocked out water molecules are higher near to the surface and are always trying to get back and to form aggregates. For water there is a high tenancy to get aggregated, and these aggregates will get back to the surface if they are not transferred from the surface by collective high velocity bombardment of air(wind!). If there are higher concentration of water molecules in air(high humidity), they could form aggregates easier and thus take more time to dry the clothes. Converse happen in less humidity.