Why am I still getting a password prompt with ssh with public key authentication?

Make sure the permissions on the ~/.ssh directory and its contents are proper. When I first set up my ssh key auth, I didn't have the ~/.ssh folder properly set up, and it yelled at me.

• Your home directory ~, your ~/.ssh directory and the ~/.ssh/authorized_keys file on the remote machine must be writable only by you: rwx------ and rwxr-xr-x are fine, but rwxrwx--- is no good¹, even if you are the only user in your group (if you prefer numeric modes: 700 or 755, not 775).
  If ~/.ssh or authorized_keys is a symbolic link, the canonical path (with symbolic links expanded) is checked.
• Your ~/.ssh/authorized_keys file (on the remote machine) must be readable (at least 400), but you'll need it to be also writable (600) if you will add any more keys to it.
• Your private key file (on the local machine) must be readable and writable only by you: rw-------, i.e. 600.
• Also, if SELinux is set to enforcing, you may need to run restorecon -R -v ~/.ssh (see e.g. Ubuntu bug 965663 and Debian bug report #658675; this is patched in CentOS 6).

¹ _{Except on some distributions (Debian and derivatives) which have patched the code to allow group writability if you are the only user in your group.}

If you have root access to the server, the easy way to solve such problems is to run sshd in debug mode, by issuing something like /usr/sbin/sshd -d -p 2222 on the server (full path to sshd executable required, which sshd can help) and then connecting from the client with ssh -p 2222 user@host. This will force the SSH daemon to stay in the foreground and display debug information about every connection. Look for something like

debug1: trying public key file /path/to/home/.ssh/authorized_keys
...
Authentication refused: bad ownership or modes for directory /path/to/home/

If it isn't possible to use an alternative port, you can temporarily stop the SSH daemon and replace it with one in debug mode. Stopping the SSH daemon does not kill existing connections so it is possible to do this through a remote terminal, but somewhat risky - if the connection does get broken somehow at a time when the debug replacement is not running, you are locked out of the machine until you can restart it. The commands required:

service ssh stop
/usr/sbin/sshd -d
#...debug output...
service ssh start

(Depending on your Linux distribution, first / last line might be systemctl stop sshd.service / systemctl start sshd.service instead.)

Is your home dir encrypted? If so, for your first ssh session you will have to provide a password. The second ssh session to the same server is working with auth key. If this is the case, you could move your authorized_keys to an unencrypted dir and change the path in ~/.ssh/config.

What I ended up doing was create a /etc/ssh/username folder, owned by username, with the correct permissions, and placed the authorized_keys file in there. Then changed the AuthorizedKeysFile directive in /etc/ssh/config to :

AuthorizedKeysFile    /etc/ssh/%u/authorized_keys

This allows multiple users to have this ssh access without compromising permissions.