How to disconnect hung ssh session in Linux. disconnect stuck ssh session in Unix, Terminate stuck ssh session. kill an unresponsive ssh session in Unix. Kill stuck ssh session. Terminate stuck ssh session in Linux. kill ssh session in Linux. Close ssh connection. Terminate an unresponsive ssh session in LInux. Automatically kill or disconnect hung ssh session using ServerAliveInterval. Automatically kill a PSSH session using timeout in Linux. Disconnect hung pssh session in linux after certain time period. Kill stuck ssh session automatically in Linux. Automatically disconnect hung ssh session in Unix.
In this article we will cover various examples around the topic to kill stuck ssh session i.e to disconnect hung ssh session in Linux, but if you wish to automatically disconnect an idle session then I have written another article wherein I have shared the steps and examples to automatically disconnect an idle ssh session in Linux.
What is an unresponsive SSH session?
By unresponsive SSH session we mean that a SSH connection is not responding or has become unresponsive between the host and the client. This situation can arise due to various reason, most likely due to network fluctuations between the server and the client. You can list all the active SSH connections and then check the idle time to get their current status. You can also get their PID from the remote host to manually kill the unresponsive ssh session.
Terminate stuck ssh session
There is a “secret” keyboard shortcut to force an unresponsive ssh session to exit.
From the frozen session terminal, hit these keys in order:
. The tilde (only after a newline) is recognised as an escape sequence by the ssh client, and the period tells the client to terminate it’s business without further ado.
From the man page of SSH
ESCAPE CHARACTERS When a pseudo-terminal has been requested, ssh supports a number of functions through the use of an escape character. A single tilde character can be sent as ~ or by following the tilde by a character other than those described below. The escape character must always follow a newline to be interpreted as special. The escape character can be changed in configuration files using the EscapeChar configuration directive or on the command line by the -e option. The supported escapes (assuming the default ‘~’) are: ~. Disconnect. ~^Z Background ssh. ~# List forwarded connections. ..... output trimmed
Here I have created an active session from node1 to node2 using user ‘
deepak‘. Now this session is in hung state.
node1:~ # ssh -q deepak@node2 Password: Last login: Fri May 31 10:47:39 2019 from 10.43.22.85 [deepak@node2 ~]$
Here i have pressed “
Enter” followed by
[deepak@node2 ~]$ <-- Here I pressed Enter [deepak@node2 ~]$ node1:~ # ^ | Here I pressed ~ (tilde) followed by . (dot)
As you see my hung session is terminated and I have returned back to my localhost.
Enter → ~ → .will get you back to your local session,
Enter → ~ → ~ → .will leave you in Machine1, and
Enter → ~ → ~ → ~ → .will leave you in Machine2. This works for other escape sequences as well, such as moving the ssh session to background temporarily. The above works for any level of nesting, by just adding more tilde’s.
Here I have made an active SSH session from node1 to node2 and from node2 to node3 i.e.
node1:~ # ssh -q deepak@node2 Password: Last login: Fri May 31 11:04:17 2019 from node1 [deepak@node2 ~]$ ssh -q firstname.lastname@example.org Password: Last login: Sun May 26 13:55:55 2019 from node2
Now here I will disconnect stuck SSH session from node3 to node2 instead of node1. So I will hit
[deepak@node3 ~]$ <-- Here I pressed Enter [deepak@node3 ~]$ [deepak@node2 ~]$ <-- And I have been disconnected from node3 and my session is at node1 -> node2 ^ | Here I pressed ~ ~ (tilde) two times followed by . (dot)
Automatically disconnect hung SSH session
Now in the above example you were manually terminating the stuck ssh session, but to automatically disconnect hung SSH session or when you are automating a task and your SSH process gets stuck, in such cases it is better if the SSH itself disconnect hung SSH session itself rather than someone manually intervening and killing the process.
In this case you can use “
ServerAliveInterval” to automatically disconnect the hung SSH session.
From the man page:
ServerAliveCountMax Sets the number of server alive messages (see below) which may be sent without ssh(1) receiving any messages back from the server. If this threshold is reached while server alive messages are being sent, ssh will disconnect from the server, terminating the session. It is important to note that the use of server alive messages is very different from TCPKeepAlive (below). The server alive messages are sent through the encrypted channel and therefore will not be spoofable. The TCP keepalive option enabled by TCPKeepAlive is spoofable. The server alive mechanism is valuable when the client or server depend on knowing when a connection has become inactive. The default value is 3. If, for example, ServerAliveInterval (see below) is set to 15 and ServerAliveCountMax is left at the default, if the server becomes unresponsive, ssh will disconnect after approximately 45 seconds. ServerAliveInterval Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server.
ServerAliveInterval you can set a timeout value which will be used if the provided SSH session becomes unresponsive. You can use this either via command line or by creating a
ssh_config file inside the respective user’s home directory.
While initiating a SSH connection use “
-o ServerAliveInterval=XX” along with the SSH command. here replace “
XX” with your suitable timeout value.
I am running a SSH session from node1 to node2, and will disconnect the NIC interface. Now this is not the best scenario to have a hung session but this will do the trick to explain our steps and example. Now here I have provided an interval of 5 seconds and TCP packet will be sent two times before disconnecting the session.
node1:~ # ssh -vvv -i /export/home/sufuser/.ssh/id_rsa -o ServerAliveInterval=5 -o ServerAliveCountMax=2 root@node2 "/sbin/ifdown bond0"
From the debug logs printed on the screen, there are two debug packets which are sent before disconnecting the session
debug3: send packet: type 80 debug3: send packet: type 80 debug3: send packet: type 1 packet_write_wait: Connection to node2 port 22: Broken pipe
Now here as expected my session my session was automatically disconnected after sending 2 TCP packets at an interval of 5 seconds.
You can configure these values per user by creating a new file
ssh_config inside user’s home directory. For example if my user’s home directory is
/home/deepak/ then create following file
# touch /home/deepak/.ssh/ssh_config
Add below content
Host * ServerAliveCountMax 3 ServerAliveInterval 10
Here * is used to refer for any matching host. If you wish to restrict it for a certain host then provide the IP/hostname.
Lastly you can update
/etc/ssh/ssh_config and add the same values. This was the configuration will not be at user level instead it will be at system level
Add below content
# vim /etc/ssh/ssh_config Host * ServerAliveCountMax 3 ServerAliveInterval 10
Use timeout to kill SSH session
You can additionally use timeout command to disconnect hung SSH session. To define a timeout value for the active session.
Here the benefit is that if for automation you are using ssh command and you do not wish the automation script to get stuck during execution then you can use timeout command to kill the session after certain period of time.
From the man page of timeout:
-k, --kill-after=DURATION also send a KILL signal if COMMAND is still running this long after the initial signal was sent
I have created a script to trap SIGTERM and capture the progress of timeout
node1:~ # cat /tmp/script.sh #!/bin/bash trap 'echo "`date`: SIGTERM received!"' 15 echo "`date`: Starting" sleep 20 echo "`date`: Stage Two" sleep 20 echo "`date`: Finished"
-k 12 switch tells timeout to send a SIGKILL to the process 12 seconds after the initial SIGTERM. Again, the “
SIGTERM received” message is displayed after 3 seconds as a result of the SIGTERM signal. Twelve seconds after that, the whole script is killed. It does not complete the second
20-second sleep, and it does not display the
Finished message after it. This is a more forceful way of dealing with the timeout.
node1:~ # timeout -k 12 3 /tmp/script.sh ; date Fri May 31 13:26:45 IST 2019: Starting Terminated Fri May 31 13:26:48 IST 2019: SIGTERM received! Fri May 31 13:26:48 IST 2019: Stage Two Killed Fri May 31 13:27:00 IST 2019
Similarly you can utilise timeout with SSH command. Assuming you know that your SSH command execution will take 10 seconds then you can define a timeout of 20 seconds as the hard period for killing the session
node1:~ # time timeout -k 25 20 ssh -q -i /export/home/sufuser/.ssh/id_rsa root@node2 "sleep 1m" real 0m20.008s user 0m0.012s sys 0m0.008s
Disconnect PSSH session
PSSH is another tool to perform SSH (parallel) instead of sequential. Now in PSSH there is an internal argument which can be used to kill the active session after a certain time period. You need not be dependent on any additional tool to disconnect the SSH session.
From the man page of pssh:
-t timeout --timeout timeout Make connections time out after the given number of seconds. With a value of 0, pssh will not timeout any connections.
Execute pssh using below syntax to perform a timeout automatically after 30 seconds
node1:~ # pssh --timeout 30 -l root -H node2 sleep 40  14:21:48 [FAILURE] node2 Timed out, Killed by signal 9
So as expected the PSSH session was automatically killed with the timeout signal after the provided timeout value.
Lastly I hope the steps from the article to disconnect hung SSH session or unresponsive SSH session on Linux was helpful. So, let me know your suggestions and feedback using the comment section.