Is "humming" normal in an AC relay?
Figure 1. Source: Machine Design.
The relay coil, when energised, pulls in the armature to actuate the contacts. Since the coil is powered by alternating current the magnetic field collapses to zero at each mains crossing and the armature tends to start to release. Its inertia is high enough that the contacts remain actuated long enough to maintain contact through each zero-cross of the mains.
The buzz is normal. It is caused by the vibration of the armature on the yoke on each half cycle.
Just a note on relay terminology: "I have a 240V relay that uses a 120VAC coil" is a little confusing. "I have a relay with 240 V contacts and a 120 V AC coil" would be clearer.
Update:
Spehro and Tony's answers both address the use of 'shading' poles on the armature to help maintain force during zero-cross. This in turn will reduce the vibration.
Figure 2. The yoke of an AC 'contactor' (high-powered relay generally used for AC motor circuits, etc.) showing two shaded poles. Image source.
AC-coil relays use a shading ring that helps smooth out the force from the varying field. Photo from this website.
The shading ring acts as a shorted turn that causes an out of phase current so the total field does not drop to zero. It still varies though.
It's normal for AC-coil relays to hum acoustically at a multiple of mains frequency. They can excite resonances in mechanical bits causing the noise to become objectionable. Large industrial contactors can make quite a racket.
DC-coil relays do not generally hum noticeably even when carrying AC (though if the contact currents are very large there can be some effect from the magnetic fields.