Analyze TLS and mTLS Authentication with Wireshark


Wireshark

Author: Celal Dogan
Reviewer: Deepak Prasad

Analyzing TLS and mTLS Protocols

In this article, we will cover Mutual Transport Layer Security (mTLS). The protocol provides a method for mutual authentication. Unlike general usage of TLS which only authenticates the server and secure the data on the fly, it enables the server to authenticate the client as well. Before diving deep, we need to be more familiar with Transport Layer Security (TLS).

 

Different TLS Record Types

TLS is a layered protocol and the bottom layer is the Record Protocol, which sends blocks of data, called records, between the client and the server. The records are in a specific format.

There are five record types for TLS version 1.0, TLS version 1.1, and TLS version 1.2:

  • Handshake
  • Change Cipher Spec
  • Data (application data)
  • Alert (Warning or Fatal Error)
  • Heartbeat

Analyze TLS and mTLS Authentication with Wireshark

TLS 1.3 specifies only three record types:

  • Handshake
  • Data (application data)
  • Alert (Warning or Fatal Error)

Following figure shows various record types, which captured during a TLS protected conversation between the client and the server.

Analyze TLS and mTLS Authentication with Wireshark

 

Each TLS record contains the following information:

  • Content type
  • Protocol version number
  • Length
  • Data payload

Following figure shows the information mentioned above.

Analyze TLS and mTLS Authentication with Wireshark

 

Understand TLS Handshake Record Content Types

Client Hello

TLS handshake starts with a Client Hello Record which includes:

  • Version: The version field is the maximum version supported by the client implementation.
  • Random: it is used later with other parameters to generate the key for encryption.
  • Session ID: It is used to resume the previous session.
  • Cipher Suites: it is a list of ciphers to be used in order of preference of use.
  • Server Name Indication: It is an extension to the TLS protocol and allows a client to indicate which hostname it is trying to connect to.
  • Application Layer Protocol Negotiation: It is a TLS extension that allows the application layer to negotiate which protocol to use.
  • Supported Version: This extension is used by the client to indicate which version of TLS the server supports.
  • Signature Algorithms: This extension indicates which signature algorithms may be used in digital signatures.

 

Server Hello

The server agrees on the cipher suit, the TLS version etc. with Server Hello response.

  • Version: The version field is the version the server prefers.
  • Random: it is used later with other parameters to generate the key for encryption.
  • Session ID: It is used to resume the previous session.
  • Cipher Suite: The cipher suite picked by the server.

 

Certificate

The server sends its certificate chain. The certificate can be used to authenticate the server.

 

Server Hello Done

The server sends Server Hello Done message, indicating that the hello-message phase of the handshake is complete.

 

Client Key Exchange

Depending on the key exchange method:

The client uses server public key to encrypt premaster key and sends encrypted premaster key to the server when RSA is used as key exchange method.

The client exchanges the Diffie Helman parameters when Diffie Helman algorithm is used as key exchange method.

 

Finished

The Client and the server notify each other to verify the negotiated options for the session.

 

Change Cipher Record Content Type

Change Cipher record has only one content type, which is called Change Cipher. The client and the server use this message to initialize the negotiated options for all future message they will send.

 

Analyze TLS Handshake with Wireshark

A typical TLS (TLS version 1.2) handshake is summarized below, assuming RSA key exchange used.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-1: The client starts a new handshake with a Client Hello and submits its capabilities. As seen below, the Client Hello packet contains cipher suits it supports, the host (info.cern.ch) it wants to connect, the application protocol it will protect (http/1.1), the TLS versions and signature algorithm it supports, etc. Since the client support, both TLS version 1.2 and 1.3, it sends the cipher suits in two different formats:

TLS version 1.2 cipher suit format is like below:

Analyze TLS and mTLS Authentication with Wireshark

 

TLS version 1.3 cipher suit format is like below:

Analyze TLS and mTLS Authentication with Wireshark

 

There are some changes in TLS 1.3 cipher suit format. The cipher suit string does not include key exchange and signature algorithm anymore. Instead it carries this information in extension “supported group” and “signature algorithm”.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-2: The server picks the cipher suit and sets the other options.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-3: The server sends its certificate and completes negotiations at the its side with Server Hello Done message.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-4: The client uses server public key to encrypt the generated premaster key and sends it to the server. After that, the client starts the encryption and informs the server about it and the last step for the client is to verify the handshake messages it sent and received.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-5: The server starts the encryption and informs the client. Then, the server verifies the handshake messages it sent and received.

Analyze TLS and mTLS Authentication with Wireshark

 

 

 

What is mTLS?

mTLs is not a new protocol and has been part of TLS specification since it was called Secure Sockets Layer (SSL). We can say that it is just a modified version of TLS. The protocol provides mutual authentication (two-way authentication), which refers to two parties authenticating each other at the same time.

 

Analyze mTLS Handshake with Wireshark

Since mTLS is just a part of TLS protocol, TLS handshake is almost the same except a couple of differences. We will use “client.badssl.com” link to test and investigate mTLS handshake.

 

Step-1:  Open your web browser and type “client.badssl.com” while capturing packets with Wireshark.

I got the following error in the browser.

Analyze TLS and mTLS Authentication with Wireshark

 

During the TLS handshake, the server (client.badssl.com) asked the client to authenticate itself with a certificate, since the client did not provide any certificate to authenticate itself, the server sent this error. We can see the same result in the packet capture as well. As seen below, the mTLS handshake is similar to TLS handshake except that server asking the client to provide a certificate in packet number 9.

Analyze TLS and mTLS Authentication with Wireshark

 

The client provides no certificate to the server in the packet number 11 and as a result the server responses with “HTTP 400 Bad Request” to the client request.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-2: To fix this issue, we need a certificate to use at the client side. Thanks to badssl.com, we can download it form here (https://badssl.com/download/). I downloaded badssl.com-client.p12 certificate and imported it to my web browser (Firefox) like below.

Analyze TLS and mTLS Authentication with Wireshark

 

Step-3: After adding the certificate to the browser, I visited the same link again and got the pop up below.

Analyze TLS and mTLS Authentication with Wireshark

 

The browser asked me to select a certificate to authenticate myself. Once I pick the certificate I imported, I was able to view the site like below.

Analyze TLS and mTLS Authentication with Wireshark 

When we take a look at the capture file, this time we are able to see that the client provided the certificate to the server.

Analyze TLS and mTLS Authentication with Wireshark

 

Final thoughts

mTLs is not a new protocol and has been part of TLS since SSL. It uses the same protocols and technologies, it’s just a mutual (two-way) verification instead of one way. mTLS helps ensure that traffic is secure and trusted in both directions between a client and server.

 

References

https://ldapwiki.com/wiki/Record%20Protocol
https://www.ibm.com/docs/en/ztpf/1.1.0.15?topic=sessions-ssl-record-format
https://www.f5.com/labs
https://wott.io/blog/tutorials/2019/09/09/what-is-mtls
https://en.wikipedia.org/wiki/Mutual_authentication

 

Celal Dogan

Celal Dogan

He is proficient in System Administration, Python, Computer Network, Network Engineering, PHP, Web Testing, Penetration Testing, Wireshark, RADIUS, Cisco Router, TCP/IP, Kali Linux, OSPF, NPS, and Multiprotocol BGP. You can connect with him on his LinkedIn Profile.

Can't find what you're searching for? Let us assist you.

Enter your query below, and we'll provide instant results tailored to your needs.

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

Buy GoLinuxCloud a Coffee

For any other feedbacks or questions you can send mail to admin@golinuxcloud.com

Thank You for your support!!

Leave a Comment