Preventing dictionary attacks on a web application

I've always been a fan of your option 3 - locking out or throttling a client based on its IP address. The other options are all more trouble than they're worth for the reasons you've stated.

Spoofing an IP address is possible, but it does not defeat this counter-measure. If you mean "spoofing" in the technical sense - forging the TCP packet header - then that won't do the attacker much good, because even if they guess the correct password they won't receive the response that tells them so. They could still use proxies, of course, but the number of proxies is limited. Even if an attacker has 1,000 working proxies at his disposal and you allow 10 attempts per IP that's 10,000 attempts. If you enforce any password complexity at all (such as requiring an alphanumeric password) then this won't be enough to guess much.

That alone should be enough to stop most script kiddies. If you are up against a more determined attacker you would probably have to implement some sort of site-wide monitoring which detects that there are many attempts being made (so there's probably an attack going on) and "locks down" in some way, eg. by using CAPTCHAs. I'm not a fan of using CAPTCHAs all the time - they're just more annoyance than they're worth.

Ultimately, it's up to the user to choose a secure password (though you can help them). If they've chosen "Password1" as their password then nothing you can do will stop a hacker from breaking into their account.

I like gmail's anti-brute force system a lot. It is based on "heat" that a user can accumulate, after the user has overheated they are prompted with a captcha. You can keep track of heat using a sql database, or using redis incr. Heat is assigned to an ip address. It is 100% impossible to "spoof" a tcp connection over the internet because of the three-way-handshake, however proxy servers are plentiful and ip address are very cheap. Proxy servers are commonly used to send spam, you can check a blacklist and automatically prompt them for a captcha.

Each bad action against your system will update the heat table. For instance a failed login will accumulate 35% heat. Once the heat level is greater than or equal to 100% then the user is forced to solve a captcha. Solving a captcha will "cool down" that ip address. The heat table could contain a timestamp column that is set to the current time on update. After 24 hours or so the heat can return to 0.

reCaptcha is the most secure captcha you can use.