How do I connect to ssh with a different public key?

Assuming you're on a Unix/Linux environment, you can create or edit the file ~/.ssh/config.

That config file allows you to establish the parameters to use for each host; so, for example:

Host host1
  HostName <hostname_or_ip>
  IdentityFile ~/.ssh/identity_file1

Host Host2
  HostName <hostname_or_ip2>
  User differentusername
  IdentityFile ~/.ssh/identity_file2

Note that host1 and host2 can also be not hostnames, but rather labels to identify a server.

Now you can log onto the to hosts with:

ssh host1
ssh host2