git suddenly started to ask for SSH key password every time

Since you mentioned, brew, I assume you're running on a Mac. This has also happened to me and the solution was to ensure that I added the passphrase to the keychain (the Mac version of ssh-agent, automatically launched on a Mac, includes keychain support):

$ ssh-add -K

And to store the passphrase for a different key:

$ ssh-add -K /path/to/private/key/file

Specifically in my case (since I use a separate key for GitHub):

$ ssh-add -K ~/.ssh/github_rsa

To automatically load keys into the ssh-agent and store passphrases in your keychain, you need to modify your ~/.ssh/config:

Host *
 AddKeysToAgent yes
 UseKeychain yes
 IdentityFile ~/.ssh/id_rsa

I obtained this information from here:

• http://www-uxsup.csx.cam.ac.uk/~aia21/osx/leopard-ssh.html#Passphrase
• https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/

The above addresses the OP issue for ssh keys. The following is also useful for Mac users if you want to cache your HTTPS credentials as well. You can do this by using a credential helper. To tell git to use the osxkeychain helper, ensure this is added to your ~/.gitconfig (or ~/.config/git/config).

[credential]
    helper = osxkeychain

Instead of editing the file directly, you can set this entry from the command line:

$ git config --global credential.helper osxkeychain

See these links for more detail (including how to verify that your system has the osxkeychain helper installed):

• https://help.github.com/articles/caching-your-github-password-in-git/
• https://git-scm.com/book/gr/v2/Git-Tools-Credential-Storage

Seems like something went wrong with my ssh-agent.

I tried to stop it (with kill -9) and start it again (with ssh-agent), but it wouldn't fix the problem.

After I ran eval 'ssh-agent -s' it all started to work as expected again.

Still I have no idea why this happened..