Steps to troubleshoot with TTL in Wireshark with Examples

Troubleshooting With TTL

The generic definition of Time to Live (TTL) refers either life time of a network packet or expiration time of the cached data in a computer.  Since it is a broad definition, I will break it into 3 categories to explain it better.

 

TTL in Internet Protocol (IP)

It refers to the number of hops that a packet can travel through before it gets discarded by a layer 3 device. The purpose of TTL in IP headers is to prevent a network packet from circulating indefinitely. TTL is 1-byte field which can be set to maximum 255. The TTL value is decreased by 1 every time the packet passes through a layer 3 device. When it reduces to 0 (zero), the packet gets dropped and an ICMP packet is sent back to the sender.

Advertisement

Steps to troubleshoot with TTL in Wireshark with Examples

Following figure shows a DNS response with TTL of 55 in IP header coming from 8.8.8.8. The TTL value most likely set to 64 when the DNS sent the response packet in the beginning. The default TTL value varies between different operating system. It even can be used to fingerprint a device.

Steps to troubleshoot with TTL in Wireshark with Examples

   

TTL in Domain Name System (DNS)

It is set by DNS server, informing the client (resolver) how long it should keep the DNS record in its cache before requesting a new one. In the figure below, the client is querying the DNS server, asking about www.google.com.  The DNS server (8.8.8.8) sends a DNS response to the client (192.168.1.52) with multiple “A” record inside the packet. Each record includes a TTL with value of 4 which means that the client should cache the record for 4 seconds.

Steps to troubleshoot with TTL in Wireshark with Examples

 

TTL in Hyper Text Transfer Protocol (HTTP)

TTL in HTTP lets the client know how long a resource (an image, a html, a media, etc.)  should be stored locally before the client (browser) makes a new request for a copy of the resource. HTTP uses “Expires” and “Cache-Control: max-age” header fields to control caching. I have a http request and response from one of our previous articles. Let’s take a look at the packets. The following figure summarizes the flow.

Steps to troubleshoot with TTL in Wireshark with Examples

 

I used “curl” as a client to create a GET request for www.example.com. The server responded with  “Expires” and “Cache-Control: max-age” fields, notifying me how long I should keep the resource in the cache. Expires field refers a time after which the response is considered stale. Cache-Control: max-age measured in seconds. When both of the fields set in a response, the “Expires” gets ignored. The actual packets are below.

Steps to troubleshoot with TTL in Wireshark with Examples

 

Troubleshooting scenario

Before we dive into packets and fields with Wireshark, take a look at the topology below.

Steps to troubleshoot with TTL in Wireshark with Examples

 

In the topology above, there are some network devices between two parts and the client can reach the web server without any problem. Take a not on the TTL times in the incoming and outgoing packets.

Advertisement
  • Gateway: It serves as a DHCP server and a gateway with simple routing functionality.
  • Firewall: It has multiple module like Firewall, IPS, Proxy, etc. It simply does packet inspection.
  • NAT Router: As the name states, it does network address translation.

 

Step 1

Imagine that one of your client (kali) comes across a failure while trying to reach the web server and the client contacts you to solve the problem.

Steps to troubleshoot with TTL in Wireshark with Examples

 

 Step 2

You ask the client to reproduce the failure while you capture the packets between two parts. For sake of learning, I assume you already captured the packets before the failure. Because we will make a comparison of TTL values for before and after.  I captured some and they look fine as below.

Steps to troubleshoot with TTL in Wireshark with Examples

 

Step 3

We will gather some information on the working connection packets. Select a packet and expand its IP header. Right click on the “Time to Live” field and next “Apply as column”. At this point you have TTL as a column like below.

Steps to troubleshoot with TTL in Wireshark with Examples

 

 Step 4

The packets arrive the client (kali) with TTL value of 40 while it sends with 64. It is great clue for troubleshooting.

Steps to troubleshoot with TTL in Wireshark with Examples

 

Step 5

In step 2, you reproduced the issue and captured the packets below.

Steps to troubleshoot with TTL in Wireshark with Examples

When we analyze the packets, the reset packet with red color draws our attention. It appears that we have got a reset from the server. The first 3 packets show that TCP 3-way handshaking looks fine. In the packet number 4, the client sends a http GET request and wait for a response. Instead of a http response, the connection gets reset.

 

Step 6

Examine the TTL time in the reset packet carefully. The TTL time is set 254 while we expect 40, more or less. It means the packet is coming from one hop away from the client. No way, it can not be the server who sent the packet. Because it is far more away from the client. When we look at the topology, the firewall is one hop away. It is obvious that the firewall did not like the response packet and reset the packet. Security devices often reset connection due to security reasons. The topology below summarizes the case.

Steps to troubleshoot with TTL in Wireshark with Examples

 

Emulate the Scenario in GNS3

I emulated the scenario in GNS3 with Cisco routers. The old cisco routers have a feature called TCP intercept, which analyses the connection and when it notices the packet poses a threat, it drops the packet and reset the connection. All Configurations for each devices are below.

 

Configuration for Gateway

!
hostname GATEWAY
!
ip dhcp pool golinuxcloud_clients
 network 192.168.2.0 255.255.255.0
 dns-server 8.8.8.8
 default-router 192.168.2.1
!
username celal privilege 15 password 0 dogan
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
 ip ospf 100 area 0
!
interface FastEthernet0/0
 ip address 192.168.23.3 255.255.255.0
 ip ospf 100 area 0
 speed auto
 duplex auto
!
interface FastEthernet0/1
 ip address 192.168.2.1 255.255.255.0
 speed auto
 duplex auto
!
router ospf 100
 redistribute connected
!

 

Configuration for Firewall

!
hostname Firewall
!
ip tcp intercept list intercept
ip tcp intercept connection-timeout 1
!
interface FastEthernet0/0
 ip address 192.168.12.2 255.255.255.0
 ip ospf 100 area 0
 speed auto
 duplex auto
!
interface FastEthernet0/1
 ip address 192.168.23.2 255.255.255.0
 ip ospf 100 area 0
 speed auto
 duplex auto
router ospf 100
!
ip access-list extended intercept
 permit ip host 192.168.2.3 any
!

 

Configuration for Nat Router

!
hostname NAT_Router
!
username celal secret 5 $1$WNlY$CTfihi7W3ZR6.Q6co0r4m/
!
!
class-map match-all http
match protocol http
!
policy-map DROP_HTTP
class http
police 8000 conform-action drop  exceed-action drop  violate-action drop
!
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
ip ospf 100 area 0
!
interface FastEthernet0/0
ip address 192.168.12.1 255.255.255.0
ip nat inside
ip ospf 100 area 0
speed auto
duplex auto
service-policy output DROP_HTTP
!
interface FastEthernet0/1
ip address dhcp
ip nat outside
speed auto
duplex auto
!
router ospf 100
default-information originate always
!
ip nat inside source list NAT_LIST interface FastEthernet0/1 overload
!
ip access-list extended NAT_LIST
permit ip any any
!

Since I did not capture the packets from a real life (production environment), I had to emulate the scenarios with dropping the packets using QoS rules, triggering the Firewall to drop the connection after one second later. When the firewall intercepted the connection, it reset the TTL time which was not compatible with real life experience. Thus, I had to modify the TTL field and some packets with Wireshark to make it look like it was from production.

 

Final thoughts

Due to security reasons, some devices in your network can interrupt or reset TCP connection. Being able to examine TTL helps you to troubleshoot in cases like that.

 

Didn't find what you were looking for? Perform a quick search across GoLinuxCloud

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

Buy GoLinuxCloud a Coffee

For any other feedbacks or questions you can either use the comments section or contact me form.

Thank You for your support!!

Leave a Comment

X