Apple - How to use SSH keys and disable password authentication
I was editing the wrong configuration file! Instead of /etc/ssh_config, I edited private/etc/sshd_config. I think this probably would have also worked if I edited /etc/sshd_config as per the updated answer from @GhostLyrics, but I didn't test that yet so I can't say for sure. After that, I restarted the service with sudo launchctl stop com.openssh.sshd and then sudo launchctl start com.openssh.sshd and I was able to get my desired behavior. Here is the resource where I found the pertinent information: https://superuser.com/questions/364304/how-do-i-configure-ssh-on-os-x
Here are the config options I changed:
PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
After that I was successfully able to generate SSH keys on my client computer, moved the public key to ~/.ssh/authorized_keys on the Mac and set permissions for that file to 644.
It is important to note that those permissions are for my public key. My private key permissions are set to 600 on my client computer. This is really important if you have both your public and private key in your ~/.ssh folder and there are multiple users on the system. If your private key permissions are set to 644 then any user could read your private key and impersonate you. Also, the permissions for the ~/.ssh folder should be 700.
/etc/ssh/ssh_config is the configuration file for the client which is used if you don't have a more specific one in your home directory. What you want to edit is /etc/ssh/sshd_config which is the one for the server.
You will probably want to set PermitRootLogin without-password (or no) and PasswordAuthentication no there.
Update:
Since you are running Yosemite, the file is /etc/sshd_config according to this answer: https://apple.stackexchange.com/a/167405/11135
To further elaborate why it still prompts when setting PasswordAuthentication no in /etc/ssh/ssh_config it is important to understand what you configured. "When making an outgoing connection via SSH, don't offer password authentication."