How secure are my passwords in the hands of Firefox using a Master Password?
In short - Firefox uses triple DES in CBC mode with Master Password.
More details: nice article about this topic is here: http://luxsci.com/blog/master-password-encryption-in-firefox-and-thunderbird.html and if you want some more details, here is mozillaZine article: http://kb.mozillazine.org/Master_password. This article gives you a detailed comparison between the major browsers.
It is believed that it is safe to store passwords such way, however, I do not trust any software. Maybe it sounds too paranoid, but we can never know where the vulnerability hides.
In order to answer "How secure are my passwords in the hands of Firefox using a Master Password?" If Firefox has any exploitable bug, then it is not secure no matter how much encryption is wrapped around your passwords. Had the question started with "Assuming Firefox is the most secure browser available in terms of exploits and ignoring any plugins..." then i would agree the answer may be irrelevant. If a click interface that is not susceptible to keystroke logging is used, then the passwords may still be safe even with a keystroke-logger in the browser. If there is a "man-in-the-browser", physically typing in a password can be intercepted and if Mozilla emulates key-presses then these would also be intercepted. If Firefox uses more direct memory access, i still would not be surprised if it could be intercepted.
"Man-in-the-browser" is not a machine level rootkit, but at the application level. Most common is malicious AJAX which can easily listen to every keystroke -- that is AJAX 101. Or could be malicious binary code injected remotely into the browser or into a plugin.
To mitigate "man-in-the-browser", use several different firefox profiles siloed for banking, email accounts, ClipperZ, and others or just use Qubes-OS.
If there is a "man-in-the-machine" or system level rootkit, then all your passwords are owned no matter if they are stored in KeePassX or ClipperZ or Firefox.