Login with SSH authorized key with changed SSH port

You can specify a non-default port on the ssh client command line using the -p option. From man ssh:

 -p port
         Port to connect to on the remote host.  This can be specified on
         a per-host basis in the configuration file.

You may wish to put both the port number and the identity file location for the host in a ~/.ssh/config file so that they don't need to be specified every time on the command line.

Ex.

Host myremotehost
  Hostname      555.555.555.555
  User          user
  Port          20002
  IdentityFile  /Users/myuser/.ssh/vpsssh

Then you will be able to use:

ssh myremotehost

Note that ssh accepts commands in the URI form, such as ssh://[email protected]:<port>. Based on that, what I do when logging in to a remote server with a private key is the following:

ssh -i ~/.ssh/id_rsa ssh://myuser@domain_name.com:2222