In order to prevent a man-in-the-middle attack, when ssh makes a connection, it saves the computer's IP and host key in the ~/.ssh/known_hosts file and the next time you try to connect, it checks the IP in that file to make sure the host key is the same.
Let me illustrate. Let's say my home IP address is 123.45.54.321. I have 2 devices, comp1 and comp2 running ssh on port 111 and 222, respectively. And let's just say their host keys are AAA and BBB (the keys are actually much, much longer). If I ssh into 123.45.54.321 port 111, then this gets entered into the known_hosts file: 123.45.54.321 AAA.
Now, if I try to ssh into comp2 at 123.45.54.321 port 222, it gets the host key BBB from the computer and looks in the known_hosts file and says, "whoa, I expected AAA from the computer at 123.45.54.321, something is wrong" then the connection fails.
There are normally a couple of solutions for this. First, you can sometimes connect to comp1, copy the key from known_hosts then remove it from the file and connect to comp2. Now put the comp1 key back in known hosts. I've never done this and heard that it doesn't always work, but sometimes ssh will keep trying entries in known_hosts until it gets one right.
The better solution is to set up ssh config file like so:
Host comp1
Hostname comp1.somedomain.com
Port 111
HostKeyAlias = comp1
CheckHostIP = no
Host comp2
Hostname comp2.somedomain.com
Port 222
HostKeyAlias = comp2
CheckHostIP = no
Then you can connect with the command ssh comp1 or ssh comp2.
Unfortunately, that doesn't work with Android because it doesn't have an ssh config file. I'm not entirely sure you can do anything about it with the normal Android ssh program. I deleted it long ago and use the ssh binary from Better Terminal Emulator Pro (see this post) and there is a workaround with it.
The first trick doesn't work with the BTEP ssh binary, and neither does the second trick because it doesn't seem to use a config file. As often is the case with Android, the ugliest solution is the only solution.
BTEP looks for the known_hosts file in /data/data/com.magicandroidapps.bettertermpro/home/.ssh (sorry, I originally posted the wrong directory. Fixed now!). You can use the first trick to get a copy of the host keys, then write a script to overwrite known_hosts depending on which computer you want to connect to.
#! /system/bin/shYou would fill in your user, path to the ssh key and port number. You can also fill in the IP address, but that's what the $(hip) part does. hip is a script that tells me my home computer's IP address which I've blogged about here and here. Essentially, I keep my home IP address in a file and the hip command just displays that file. That way, I don't need to hard code my IP in any scripts. If my IP changes, I can just update it in that file and not worry about changing the scripts.
case "$1" in
comp1)
echo "$(hip) ssh-rsa AAA==" > /data/data/com.magicandroidapps.bettertermpro/home/.ssh/known_hosts
ssh USER@$(hip) -i PATH/TO/KEYFILE -p PORT;;
comp2)
echo "$(hip) ssh-rsa BBB==" > /data/data/com.magicandroidapps.bettertermpro/home/.ssh/known_hosts
ssh USER@$(hip) -i PATH/TO/KEYFILE -p PORT;;
esac
I named the script above simply "s". So, if I run s comp2, it will write 123.45.54.321 ssh-rsa BBB== to known_hosts then connect to comp2.
Incidentally, this also takes care of another problem- IP changes. The first time you connect to a host, ssh asks if you really want to connect and you have to type yes or no.
If you normally have 123.45.54.321 ssh-rsa BBB== in your known_hosts file and your home IP changes for some reason, the new IP won't be in the file and your scripts will break until you manually log in and tell ssh yes or no. But now that no longer matters because we are controlling known_hosts and filling it with the correct up-to-date IP and associated key.
The other solutions are better for computers that are running the real OpenSSH, but this is the only solution I could figure out for Android and it's funky programs.