Im gonna fly into an example so you can see what actually happens when you bang out a traceroute command on a router. In the network below, Im going to traceroute from Sw1 to Sw2.
Sw1#traceroute 24.24.24.2
Type escape sequence to abort.
Tracing the route to 24.24.24.2
1 13.13.13.3 0 msec 0 msec 9 msec
2 34.34.34.4 0 msec 0 msec 0 msec
3 24.24.24.2 8 msec * 0 msec
Sw1#
A wireshark capture was taken on Sw1, and this can be viewed online here (I recommend you open this file before reading on).
Traceroute can be explained in three main steps below.
1- Traceroute starts by sending 3 UDP packets with a TTL set to 1, towards the destination. Each UDP packet gets an ICMP reply with a Time-to-live exceeded message. The key thing is that the IPv4 Source field is now filled with an IP address (13.13.13.3 line 2 in the packet capture). This means 13.13.13.3 is now our first hop.
2- Sw1 then sends another 3 UDP packets to the destination with a TTL of 2. So the packet goes past the first hop (Sw3), gets to the second hop (Sw4), and another 3 ICMP TTL exceeded messages are sent back with the source IP field filled in (34.34.34.4 line 8 in the packet capture). This means 34.34.34.4 is our second hop.
3- Sw1 then sends another 3 UDP packets to the destination with a TTL of 3. This time it actually reaches the destination, and we get an ICMP Destination unreachable (Port unreachable) message back. Because we can actually reach the destination (at layer 3) and the TTL has not been exceeded, it now tries to reach the destination port (layer 4). This verifies that this is the final hop, and a TTL of 3 meant the destination was 3 hops away. It doesnt matter that the port was unreachable, it was simply a test to get that port unreachable message back so that we know we moved up the OSI stack to layer 4, which verifies layer 3 is reachable.
The last thing is that traceroute always starts at port 33434 and increments by 1 each time a UDP packet is sent. You can see this in the packet capture. The first line shows the destination port is traceroute (which is 33434). The next red line shows 33435, then 33436 and so on.