paramiko, isn't talking to ssh-agent. same behavior in fabric
So from the paramiko code and yours when you do a.get_keys() that should return a list. I'd see what it returns. And it woudln't return something you can count like that, as it's returning the actual encrypted key bits. But anyhow, as you've moved onto ssh, and that works, let's move to Fabric.
You can get more logging by turning it on for the ssh lib by doing:
import ssh
ssh.util.log_to_file("paramiko.log", 10)
In your fabfile. This'll up all the logs and show more of what paramiko/ssh itself is doing which may assist you in debugging the issue further.
Ok, so the first thing I discovered was that Paramiko is way out of date, and unmaintained.
It's now known as package ssh, at least under Ubuntu, and has a different maintainer (bitprophet)
Here's a demo class that works exactly as described: https://raw.github.com/bitprophet/ssh/master/demos/demo.py
It requires this file, for interactive prompts: https://github.com/bitprophet/ssh/blob/master/demos/interactive.py
Here's a sample session, using it:
$ ./ssh_demo.py
Hostname: 192.168.1.10
*** Host key OK.
Username [bryan]: root
Trying ssh-agent key eee5638f390e1698898984b10adfa9317 ... success!
*** Here we go!
Linux top.secret.com 2.9.37-1-amd64 #1 SMP Thu Nov 3 03:41:26 UTC 2011 x86_64
┌┌(root@top)-(10:44am-:-03/27)┌-¨-¨¨˙
That doesn't answer the question of why fabric isn't authenticating against the ssh-agent correctly thought. So the question remains open.
Update:
Thanks to Morgan's hint, I've gotten a little further with this problem. As he suggested, I enabled ssh logging by adding the following to the top of my fabfile.py
from fabric.api import *
import ssh
ssh.util.log_to_file("paramiko.log", 10)
I also monitored the server log. In doing so I discovered that the user which I specified was being disregarded and my local username used instead.
On the server:
tail -f /var/log/auth.log
Mar 28 11:12:36 xxxxxxxxxxx sshd[17652]: Invalid user bryan from xxx.xxx.xxx.xxx
Locally:
tail -f paramiko.log
DEB [20120328-11:39:29.038] thr=1 ssh.transport: starting thread (client mode): 0x8dfc66cL
INF [20120328-11:39:29.066] thr=1 ssh.transport: Connected (version 2.0, client OpenSSH_5.5p1)
DEB [20120328-11:39:29.093] thr=1 ssh.transport: kex algos:['diffie-hellman-group-exchange-sha256', 'diffie-hellman-group-exchange-sha1', 'diffie-hellman-group14-sha1', 'diffie-hellman-group1-sha1'] server key:['ssh-rsa', 'ssh-dss'] client encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'arcfour256', 'arcfour128', 'aes128-cbc', '3des-cbc', 'blowfish-cbc', 'cast128-cbc', 'aes192-cbc', 'aes256-cbc', 'arcfour', '[email protected]'] server encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'arcfour256', 'arcfour128', 'aes128-cbc', '3des-cbc', 'blowfish-cbc', 'cast128-cbc', 'aes192-cbc', 'aes256-cbc', 'arcfour', '[email protected]'] client mac:['hmac-md5', 'hmac-sha1', '[email protected]', 'hmac-ripemd160', '[email protected]', 'hmac-sha1-96', 'hmac-md5-96'] server mac:['hmac-md5', 'hmac-sha1', '[email protected]', 'hmac-ripemd160', '[email protected]', 'hmac-sha1-96', 'hmac-md5-96'] client compress:['none', '[email protected]'] server compress:['none', '[email protected]'] client lang:[''] server lang:[''] kex follows?False
DEB [20120328-11:39:29.093] thr=1 ssh.transport: Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
DEB [20120328-11:39:29.093] thr=1 ssh.transport: using kex diffie-hellman-group1-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
DEB [20120328-11:39:29.183] thr=1 ssh.transport: Switch to new keys ...
DEB [20120328-11:39:29.224] thr=2 ssh.transport: Trying SSH agent key cda5638f390e166864444b1093b91017
DEB [20120328-11:39:29.272] thr=1 ssh.transport: userauth is OK
INF [20120328-11:39:53.310] thr=1 ssh.transport: Authentication (publickey) failed.
DEB [20120328-11:41:29.076] thr=1 ssh.transport: EOF in transport thread
Hmm, that's strange, I ran the command as: fab diskfree -H xxx.xxx.xxx.xxx -u root
But what is this?
$ cat ./fabfile.py
from fabric.api import *
import ssh
ssh.util.log_to_file("paramiko.log", 10)
env.user = 'bryan'
def host_type():
run('uname -s')
def diskfree():
run('df -h')
Hmm
env.user = 'bryan'
Could that be the root of the problem? Could the ssh error messages just be misleading me?
I removed the line and it worked, so I guess so, is the answer.