Site/App Performance

Diagnosing Network and Latency Issues with MTR

If you are experiencing issues with latency or network connection to your server, there is a powerful tool that can assist in diagnostics. While many users are familiar with traceroute — which measures the path data takes from your machine to your server — MTR is a much more powerful tool which provides more in-depth data, allowing you to more easily diagnose problems.

Installing MTR
The first step is to install MTR on your machine. You can do this on both your server as well as your home computer to provide data going into the server and coming out of it.

Windows
Installing on Windows requires you to download and install the Windows port called WinMTR. You can download it here.

Linux
To install on a CentOS or Fedora system, execute the following commands from shell:

yum install mtr

For Ubuntu and Debian, use the following:

apt-get install mtr-tiny

For Arch Linux, use the following:

pacman -S mtr

Mac
To install on a Mac, you can find packages here, though this may not be the most up to date distribution. If you want to insure the most up to date distribution, you will require either Homebrew or MacPorts.

To install with Homebrew, use the command:

brew install mtr

For MacPorts, use the command:

port install mtr

Alternately, you can install the package from here.

Generating Report
Linux and Mac
To generate a MTR report on either Linux or Mac, you need to execute the following command. For Mac, you may need to append sudo to the command because of file permissions.

mtr -rwc 500 [site]

This will generate a report of 500 packets, which is what ServInt considers the minimum for proper diagnosis of an issue. For example, to generate a report for the ServInt homepage, you would run:

mtr -rwc 500 servint.net

To generate a report from your server to your home machine, you would replace the site with your IP address. If you do not know your IP, you can find it here. For example:

mtr -rwc 500 209.50.226.20

Before you can generate this report, you must first enable outbound traceroutes from your CSF (ConfigServer Security & Firewall) in cPanel. An explanation of how to do this is found in this article.

Windows
WinMTR uses a graphical user interface. Open WinMTR and type site address into the box and select Start.

Reading and Analyzing the Report
MTR delivers a great deal more information than a typical traceroute does. Here is an example MTR run to the ServInt website.

mtrbase1

The report consists of a series of hops, which are the individual points of the Internet that the data travels through. Each hop contains several pieces of data.

The first is the domain name or IP address of each individual hop. This can be useful in telling you where a problem may arise.

mtrlocal

The second is the Loss%. This shows what percentage of the packets of data sent were lost at each hop. The Snt column shows how many individual packets were sent. This will usually be 10, unless you specify a different number with the addition of —report-cycles=[#], replacing [#] with a number of packets you want sent.

mtrloss

The next four columns all measure latency, which shows how long it takes (in milliseconds) for the data to reach the node and return. Last shows the latency of the last packet. Avg is the average time of all packets. Best displays the shortest round trip time, while Wrst displays the longest time. The Avg column is generally the one you’ll be looking at.

mtrping

StDev displays the standard deviation of the latencies. The higher the number, the greater the difference between measurements. If this number is high, it means the measurements were inconsistent (i.e., some very low and others very high).

mtrstdev

In general, the first 2 or 3 nodes are from the source’s ISP, the last 2 or 3 are the destination’s ISP, and the ones in the middle are the route taken between the two ISPs.

Packet Loss
To diagnose problems, you should first look at loss and latency. If there is a loss percentage at any hop, there could be a problem there. However, there are times when certain nodes will have built-in limits to the traffic that programs like MTR use. If the hop immediately after shows a 0% loss, it’s likely this is what’s occurring.

mtrpacket

If the loss continues, it’s likely that there is an actual packet loss or routing issue. If different amounts of loss are occurring between hops, you can generally consider the later hops to be more accurate. It is also possible that the lost packets come on the return trip, which is why it’s useful to have reports from both directions.

The last 3 hops show packet loss, indicating an actual problem.

The last 3 hops show packet loss, indicating an actual problem.

Because the Internet is designed to handle small amounts of loss, small amounts of loss around 10% or less can be safely ignored. The loss can be compensated for and is likely only temporary. Only become concerned if there is a greater amount of packet loss or it persists for an extended period of time.

Latency
The other problem that can commonly be diagnoses with MTR is network latency. The latency should increase in a fairly consistent manner between each hop. If there is a very large increase in latency times between two hops, you may be experiencing a problem.

mtrbase

Much like packet loss, latency may be caused by the return path. Also like packet loss, data limits may make it seem like you are experiencing latency. If the latency spikes only on a single node, but drops down on a later one, it is probably data limits which don’t actually affect your speed.

mtrlatency

Make sure to focus on the final hop when evaluating latency. This shows how long it takes traffic to go from the source to the destination and return, so it has the most “real world” applicability.

Common Issues
While some issues require escalating to the providers of upstream networks, others can be fixed by end users. If you wish to diagnose your problems, here are a few common issues.

100% Destination Packet Loss

mtrfinal
If the destination hop is showing 100% packet loss and has 0.0s for the latency, it’s likely that the destination is simply not sending a reply. This may be because of improperly configured networking or firewall rules. Solving this requires making changes to the destination rules. If it is your server where this is occurring, you can contact our MST for assistance.

100% Loss on ??? Hop

mtrrouter
A residential gateway (usually your router or modem) may report a 100% packet loss on a hop identified by ???. This is simply due to how they work. As long as subsequent hops report normally, you do not need to worry.

??? Hops with No Additional Issues

mtrunknown
Some routers simply discard data from programs like MTR and do not return answers. As long as the rest of the report seems in good order, you can ignore this problem.

Several ??? Hops or Repeated Hops Which Deadend

mtrloop
If several hops in a row show the same IP or ???s, without ever reaching the destination, this is because the router on the hop before the problem is configured improperly. The solution to this is to contact your ISP and report the problem, as they will then be able to report it to the proper upstream provider.

Other Issues
For the vast majority of routing issues displayed by MTR, you can be assured that the ISP’s engineers are already working on resolving it. If you are experiencing problems for an extended period, you may wish to report them. Be sure to include your MTR reports when you do, as it will show them there’s a problem!

If you are ever experiencing an issue and are unable to interpret your MTR results, feel free to open a support ticket with us. Our techs are trained to read the data and help you diagnose any issues you may be having.

Find out more about ServInt solutions
VPS

Starting at $27

  • The New York Times
  • The Hill
  • Bloomberg
  • The Seattle Times
  • Computer World
  • Ars Technica
  • MSNBC