In one of the previous articles Analyze TLS and mTLS Authentication with Wireshark, we explored how SSL/TLS handshake works and analyzed SSL/TLS record types in Wireshark. TLS handshaking employs three subprotocols that are used to allow peers to agree upon security parameters for the record layer, to authenticate themselves, to instantiate negotiated security parameters, and to report error conditions to each other. These protocols are:
- The Handshake Protocol, which performs the initial key negotiation.
- The ChangeCipherSpec Protocol, which changes the encryption system currently in use.
- The Alert Protocol, which sends important messages about the state of the SSL/TLS connection from one side to the other.
In this article, we will focus on the Alert Protocol to troubleshoot TLS issues.
This protocol uses alert messages to notify the peers about the status of the TLS connection. The messages fall into two levels (categories): Warning and Fatal.
Each alert message consists of two parts: An Alert Level and an Alert Description. Following table shows a brief description for the levels.
|Alert Description (Meaning)
|It is a warning, which does not cause a connection termination.
|Requires immediate termination of the connection.
Alert messages convey the severity of the message (warning or fatal) and a description of the alert. Alert messages with a level of fatal result in the immediate termination of the connection. Like other messages, alert messages are encrypted and compressed, as specified by the current connection state. For troubleshooting, we will analyze messages in two groups: Closure and Error Alerts.
Every TLS connection ends with this alert message, which notifies the recipient that the sender will not send any more messages on this connection. Either party may initiate a close by sending this alert. Any data received after a closure alert is ignored. Unless some other fatal alert has been transmitted, each party is required to send this alert (close notify) before closing the write side of the connection. The screenshot below shows a typical data transfer over a TLS connection.
The data transfer follows the steps below:
- The TCP connection is established with the TCP 3-way handshaking.
- For securing the data, the TLS session is created with the TLS handshake.
- The data gets transferred securely.
- No data to send and the TLS session is terminated with Closure Alert.
- The final step is closing the TCP connection.
Since this message is encrypted, we can not see the real alert message. So how do we actually know this is a closure alert? Without decrypting the message, it is not possible, but we can check the pattern above (5 steps) see if it is match. There is another way, which is decrypting the traffic. Here are some articles on decrypting TLS (https://www.golinuxcloud.com/wireshark-decrypt-ssl-tls-tutorial/ https://www.golinuxcloud.com/decrypt-rdp-traffic-wireshark-frida/ )
After decryption, we will see the output below.
This alert level is warning, which is completely normal to be seen during a connection termination. While troubleshooting, you should not focus on Closure Alerts.
Error handling in the TLS Handshake protocol is very simple. When an error is detected, the detecting party sends a message to the other party. Upon transmission or receipt of a fatal alert message, both parties immediately close the connection.
If an alert with a level of warning is sent and received, generally the connection can continue normally. If the receiving party decides not to proceed with the connection, it sends a fatal alert to terminate the connection. Given this, the sending party cannot, in general, know how the receiving party will behave. Therefore, warning alerts are not very useful when the sending party wants to continue the connection, and thus are sometimes omitted. For example, if a peer decides to accept an expired certificate (perhaps after confirming this with the user) and wants to continue the connection, it would not generally send a Certificate Expired alert.
RFC 5246 defines following error alerts:
|This notifies the recipient that the sender will not send any more
messages on this connection. Either party may initiate a close by sending this alert.
|An inappropriate message was received. This alert is always fatal and should never be observed in communication between proper implementations.
|bad record mac
|This alert is returned if a record is received with an incorrect MAC. This message is always fatal and should never be observed in communication between proper implementations.
|decryption failed RESERVED
|This alert was used in some earlier versions of TLS, and may have
permitted certain attacks against the CBC mode.
|A TLSCiphertext record was received that had a length more than 2^14+2048 bytes, or a record decrypted to a TLSCompressed record with more than 2^14+1024 bytes. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network)
|The decompression function received improper input (e.g., data
that would expand to excessive length). This message is always
fatal and should never be observed in communication between proper implementations.
|Reception of a handshake_failure alert message indicates that the sender was unable to negotiate an acceptable set of security
parameters given the options available. This is a fatal error.
|no certificate RESERVED
|This alert was used in SSLv3 but not any version of TLS.
|A certificate was corrupt, contained signatures that did not
verify correctly, etc.
|A certificate was of an unsupported type.
|A certificate was revoked by its signer.
|A certificate has expired or is not currently valid.
|Some other (unspecified) issue arose in processing the
certificate, rendering it unacceptable.
|A field in the handshake was out of range or inconsistent with
other fields. This message is always fatal.
| A valid certificate chain or partial chain was received, but the
certificate was not accepted because the CA certificate could not be located or couldn't be matched with a known, trusted CA. This message is always fatal.
| A valid certificate was received, but when access control was
applied, the sender decided not to proceed with negotiation. This message is always fatal.
|A message could not be decoded because some field was out of the specified range or the length of the message was incorrect. This message is always fatal and should never be observed in
communication between proper implementations (except when messages were corrupted in the network).
|A handshake cryptographic operation failed, including being unable to correctly verify a signature or validate a Finished message. This message is always fatal.
|export restriction RESERVED
|This alert was used in some earlier versions of TLS.
|The protocol version the client has attempted to negotiate is
recognized but not supported. (For example, old protocol versions might be avoided for security reasons.) This message is always fatal
|Returned instead of handshake_failure when a negotiation has
failed specifically because the server requires ciphers more
secure than those supported by the client. This message is always fatal.
|An internal error unrelated to the peer or the correctness of the
protocol (such as a memory allocation failure) makes it impossible to continue. This message is always fatal.
| This handshake is being canceled for some reason unrelated to a
protocol failure. If the user cancels an operation after the
handshake is complete, just closing the connection by sending a
close_notify is more appropriate. This alert should be followed
by a close_notify. This message is generally a warning.
|Sent by the client in response to a hello request or by the server
in response to a client hello after initial handshaking. Either
of these would normally lead to renegotiation; when that is not
appropriate, the recipient should respond with this alert. At
that point, the original requester can decide whether to proceed
with the connection. One case where this would be appropriate is where a server has spawned a process to satisfy a request; the
process might receive security parameters (key length,
authentication, etc.) at startup, and it might be difficult to
communicate changes to these parameters after that point. This message is always a warning.
|Sent by clients that receive an extended server hello containing
an extension that they did not put in the corresponding client
hello. This message is always fatal.
Common TLS Handshake Failures
TLS handshake failure, which may be due to many reasons, is a common error. Sometimes troubleshooting them can be a frustrating. Thanks to Wireshark with decrypting TLS feature and great display filters, we can find them easily. Some common failures are below.
A certificate expires once its validity period is over. When one side receives the expired certificate, it sends this alert message. As seen below, the client notifies the server with a fatal error and closes the connection after receiving an expired certificate.
Sometimes, this error happens due to mismatched time synchronization.
This alert message is sent when there is a certificate mismatch between the client and server. I tried to connect to “wrong.host.badssl.com”, specifying the domain name in the TLS Client Hello packet with Server Name Indication field. However, the server sent me a certificate for “badssl.com” domain. Due to this reason the client sent Bad Certificate alert message and terminated the connection.
This error happens when a valid certificate chain or partial chain is received, but the certificate is not accepted because the CA certificate can not be located or matched with a known, trusted CA. This message is always fatal, which terminates the TLS session.
Revoking a certificate is a process of invalidating an issued SSL certificate. A revoked certificate is no longer trusted and can no longer be used for certificate-based encryption. A certificate is revoked mostly when its private key shows signs of being compromised. Following screenshot shows that the certificate has been revoked.
Reception of a handshake failure alert message indicates that the sender was unable to negotiate an acceptable set of security parameters given the options available. This is a fatal error. As seen in the screenshot below, the client sends a bunch of cipher suits, none of which is supported by the server.
Since the server supports only RC4-MD5 security parameters, it responses back with a handshake failure alert.
The protocol version that one side has attempted to negotiate is recognized but not supported. (For example, old protocol versions might be avoided for security reasons.) This message is always fatal.
The following screenshot shows that the client negotiates for TLS version 1.2 whereas the server supports only version 1.0. Therefore, the client terminates the session with sending protocol version alert.
Filtering TLS Handshake Failures
There are times when we need to take a quick look at the TLS alerts for troubleshooting. Wireshark provides a display filter for this purpose. After applying “tls.record.content_type == 21” display filter, all alerts will be displayed in Wireshark like below.
With Alert Protocol, it is very easy to troubleshoot TLS handshake problems. However, some of the alerts are encrypted and we can not see the alerts details like level and message. The TLS traffic must be decrypted so that the alerts can be seen in clear text.