Fixing SSH login long delay

For a long time I had a problem with ssh login on a Redhat 6 server – it was taking too long to connect to it, around 30 seconds. Normally it hasn’t been a big issue – after all, you connect once and work for all day as long as you enable server keepalive packets to avoid session timeout.

However when it comes to work with SFTP o GIT it might become annoying. Everytime you sFTP upload or  git push you have to wait 30 seconds again.

This kind of problems are often related to DNS issues but this is not always the case. Following are the most common solutions:

1. Disable reverse IP resolution on SSH server

It turns out there is a setting in OpenSSH that controls whether SSHd should not only resolve remote host names but also check whether the resolved host names map back to remote IPs. Apparently, that setting is enabled by default in OpenSSH. The directive UseDNS controls this particular behaviour of OpenSSH, and while it is commented in sshd_config (which is the default configuration file for the OpenSSH daemon in most enviornments), as per the man page for sshd_config, the default for UseDNS is set to enabled. Add the following line:

UseDNS no

2. DNS resolver fix for IPv4/IPv6 enabled stacks

It’s a known issue on the Red Hat knowledgebase article DOC-58626, but since it’s closed without login, I’ll share the solution below:

The resolver uses the same socket for the A and AAAA requests. Some hardware mistakenly only sends back one reply. When that happens the client sytem will sit and wait for the second reply. Turning this option on changes this behavior so that if two requests from the same port are not handled correctly it will close the socket and open a new one before sending the second request.

The solution is to add the following line to your /etc/resolv.conf. Just add it all the way at the bottom, as the last line.

options single-request-reopen

3. Disable GSSAPI authentication method

OpenSSH server enables by default the GSSAPI key exchange which allows you to leverage an existing key management infrastructure such as Kerberos or GSI, instead of having to distribute ssh host keys throughout your organisation. With GSSAPI key exchange servers do not need ssh host keys when being accessed by clients with valid credentials.

If you are not using GSSAPI as a authentication mecanism, it might be causing this connection delay.

In my particular case, I ran ssh -v myserver to find out that it was hanging whilst attempting to authenticate with GSSAPI, with the slow section looking like:

....
....
debug2: key: /home/user/.ssh/id_rsa (0xb961d7a8)
debug2: key: /home/user/.ssh/id_dsa ((nil))
debug2: key: /home/user/.ssh/id_ecdsa ((nil))
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found

debug1: Unspecified GSS failure.  Minor code may provide more information

Turned out that it was stalling after trying gssapi-with-mic authentication method. Had several “Unspecified GSS failure” messages with several seconds delay between them, therefore it was definitely the root cause of long delays.

The fix is simple – disable attempts to use GSS-API by adding the following to /etc/sshd_config (server side) or yout ~/.ssh/ssh_config (client side).

GSSAPIAuthentication no

There is an easy way to check beforehand whether this solution will work. Try to ssh into your server by disabling GSSAPI authentication:

ssh -o GSSAPIAuthentication=no user@yourserver

44 thoughts on “Fixing SSH login long delay

  1. Pingback: ASMCA» Blog Archive » Fixing SSH login slow issue

  2. many thanks for this article…. I thought I’d solved by using UseDNS no, but after that I found out that I had a wrong entry in my /etc/hosts. After fixing that no hangs occur any longer.

    best tcpdump

  3. Pingback: slow ssh | My Tech Notes

  4. Pingback: SSH take long time to login on CentOS | Crackjet

  5. Thanks a lot, the terms DNS and IPV4/IPV6 were the key hints in my case.

    By default ssh seems to try an IPV6 name translation which may take a long time in some cases (before it fails). You can nail ssh to IPV4 by the ‘-4’-option or by specifying ‘AddressFamiliy inet’ in one of the ssh-config files.

  6. On newer versions of Linux that use systemd you may get slow login if you change the root username to something else. Some admins do this to stop hackers from trying to guess the root password. If you changed the root username in /etc/passwd and /etc/shadow files then you need to also change it in all the files under /etc/dbus-1/system.d. Otherwise it will cause lookup failures and cause slow logins.

  7. I noticed your website’s ranking in google’s search results is very low.
    You are loosing a lot of traffic. You need high authority backlinks to rank in top ten. I
    know – buying them is too expensive. It’s better to own them.
    I know how to do that, just google it:
    Polswor’s Backlinks Source

  8. Yet another Thanks Message
    Well, in my case, I preferred to fix my broken reverse DNS, but what matters is : you pointed me what was wrong !

  9. Thanks.. It worked for me also after adding UseDNS No in my linux server .
    In my case ,There is another issue while logging out it is still taking too much time .DO you have any handy resolution or reason for this behaviour .

  10. I’ve disabled GSSapi and reverse DNS, however with RHEL 6 clients they also try to auth using ECDSA key, which we do not use. Is there a way to disable this? There is nothing in the sshd_config file.

  11. Pingback: ssh 慢的问题 | PHPor 的Blog

  12. Awesome. Second one did it. (Had already applied first one long ago.) On Ubuntu I had to edit /etc/resolvconf/resolv.conf.d/tail and then run `resolvconf -u`.

  13. I would like to add another possible fix to this issue. If your server isn’t one that is recognized via DNS via hostname then you may need to add it’s own entry to it’s own /etc/hosts file.
    ex:
    10.10.10.123 myserver.mydomain myserver

    Once I did this, my problem disappeared.

    • To further clarify, you would be attempting to SSH to your server via hostname and probably not by just IP address for this particular fix to apply. (in which case you probably already have the hostname/IP combo defined in your client side HOSTS/HOST file)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s