Avoiding echo/feedback on speaker-phones, how?
The audio processing algorithm you are interested in is called "Acoustic Echo Cancellation", or AEC. It is most commonly used in speakerphones to remove the output of the speaker from the mic signal. Most of this benefit is to the person on the other end of the phone call, since he won't be hearing echos of himself.
Some cheap and not so cheap speakerphones don't use AEC. I have a Polycom speakerphone which is "half duplex". Meaning that when one side is talking, the other side is muted. Because of this, there is no chance for echos or feedback to happen. Unfortunately, this also allows for a "filibuster"-- if one side never shuts up then the other side can never interrupt.
There are many types of AEC algorithms, and almost every type is patented. Most of them involve some form of modeling, where a model of the "speaker to mic acoustic signal path" is created. Once created, we can predict how the speaker output will be picked up by the mic, and thus remove that signal from the mic, leaving only the intended sounds in the mic signal.
This model would thus figure out how the sounds reflect off of the walls and other things in the room, etc. The patents for AEC usually center around exactly how this model is initially created and later updated as things change in the room (mic position, position of people and furniture, etc).
In addition to the "room model", there are other noise-reduction algorithms used. While these algorithms are not technically part of AEC, there are no useful implementations of AEC that don't use these. Normally there is some sort of simple noise-gate (or a multi-band noise gate). Other algorithms are also typically used, but are either patented or treated as a "trade secret"-- which is why I can't tell you about them! :(
Most AEC algorithms operate on a limited frequency range, 300 Hz to 3 KHz, which is the same frequency range as most telephones. Increasingly, wide-band AEC is becoming popular with the advent of higher-bandwidth teleconferencing/telepresence systems.
AEC algorithms are very computationally expensive, and the wide-band AEC requires several times more horsepower than the more limited versions. It is not uncommon for a single "run-of-the-mill" DSP to only be able to do 1 or 2 channels of AEC. For a high quality wide-band AEC, a single high-powered DSP might be required for a single channel.
AEC algorithms are also very difficult to implement. In the entire USA, there are perhaps only 10 or 20 people who have the ability to write a good one. One very smart person that I know just wrote a wide-band AEC algorithm and it took him over a year!
For a 2-way baby monitor, I highly recommend using a half-duplex approach!
One approach which is very common in low-cost speakerphones is to adjust the gain downward (possibly to zero) on whichever 'direction' has the lower apparent signal at the input. Some phones will mute the microphone except when the audio level at the mic is higher than could be attributed to speaker feedback, in which case they mute the speaker. Other phones use a more adaptive approach, cutting the quieter path enough to keep the overall echo-loop gain well below one, but not so much that both parties could be talk without being aware that the other was trying to do so.
Perfect echo cancellation is hard, but combining reasonably-effective echo cancellation with adaptive attenuation of the 'quieter' signal path can give results that are, subjectively, almost as good.
This is a very late reply but I hope it helps a few. If you are a designer you could use echo cancellation IC from Microsemi, Cirrus Logic or Forte Media. You could choose your part based on your requirements from manufacturer websites. I could not find a echo cancellation IC that can be bought for less than $2 at 3000MOQ. There are low cost ARM chips which support software echo cancellation up to some level. Also if I am right many media-streamer libraries support echo cancellation.