Getting a pid for an ssh process that backgrounded itself

Finding the pid by grepping might be error prone. Alternative option would be to use ControlPath and ControlMaster options of SSH. This way you will be able to have your ssh command listen on a control socket and wait for commands from subsequent ssh calls.

Try this

ssh -D localhost:8087 -S /tmp/.ssh-aws-gateway-vpc1 -M -fN aws-gateway-vpc1
# (...)
# later, when you want to terminate ssh connection
ssh -S /tmp/.ssh-aws-gateway-vpc1 -O exit aws-gateway-vpc1

The exit command lets you kill the process without knowing the PID. If you do need the PID for anything, you can use the check command to show it:

$ ssh -S /tmp/.ssh-aws-gateway-vpc1 -O check aws-gateway-vpc1
Master running (pid=1234)

The $! doesn't work, as you say, because it hasn't been backgrounded by the current shell. In fact, the ssh process isn't even a child of the shell you launched it from. On my Arch system, at least, it is run as a child of PID 1, the init process.

So, to get the PID, you can simply use ps:

$ ssh -f  localhost sleep 100
$ ps aux | grep '[s]sh.*-f'
terdon   20648  0.0  0.0  43308   680 ?        Ss   12:15   0:00 ssh -f localhost sleep 100

That tells me the PID is 20648^*.

Alternatively, and more simply, use pgrep -f:

$ pgrep -f 'ssh.*-f'
20648

And, to kill it (them):

pkill -f 'ssh.*-f'

---

^* See this question if you're wondering about the [s] in the grep command.