Latency and throughput are crucial concepts for a Systems Design interview. Since building Microservices in the cloud is the new normal in the market, knowing basic network concepts make a big difference. We will often see those terms when designing a system in the cloud.
Let’s explore those concepts further so you get sharp for the Systems Design interview and learn to build systems in the cloud more effectively.
Latency
Latency is the time it takes for data to travel from one point to another in a system. We measure latency in nanoseconds (ns), microseconds (µs), milliseconds (ms), or seconds (s).
Latency is significant because it affects the performance of a system. For example, if a website has high latency, it will take longer for users to load and interact with it.
What can affect latency?
Several factors can affect latency, including:
- The distance between the two points: The further apart the two points are, the higher the latency will be.
- The type of network: A wired network typically has lower latency than a wireless network.
- The amount of traffic on the network: If there is a lot of traffic, latency will be higher.
What can reduce latency?
Latency can be reduced by using a wired network, reducing the distance between the two points, and using a network that is not congested.
Simulation of Latency Time for Data Reading
Let’s roughly estimate the is the latency to transfer one megabyte of data:
| Reading Data from | Time in Milliseconds |
|---|---|
| SSD (solid-state drive) | 1 ms |
| Via Network (If it’s in the same country or close enough) | 10 ms |
| HDD (hard disk drive) | 20ms |
| Via Network From Ireland to the US | 150ms |
Notice that sending 1 MB via a network is faster than reading data from HDD. Also, sending data from one continent to the other is much slower. Your systems design decisions have to take that in consideration if your system needs fast response.
For example, the latency from Ireland to the US is roughly 150 ms for each MB. This is relatively fast for a single packet of data. However, if you send a large file, the latency will increase, and the transfer will take longer.
Why latency is important
Latency is important for gaming or any application that requires quick responses. In a game, for example, it is important to have low latency so that the game is responsive and players can react quickly. If the latency is too high, it can make the game unplayable.
For websites, latency might not be as important as availability. This is because users can usually wait a few seconds for a website to load. However, if a website is unavailable, users cannot access it.
In general, latency is important to consider when designing and building a system. You can design a fast and reliable system by understanding the different factors that affect latency.
Throughput
Throughput is the rate at which data is successfully transferred from one point to another. We typically measure throughput in bits per second (bps), kilobits per second (Kbps), or megabytes per second (Mbps).
Throughput is crucial because it determines how quickly we can transfer data between two devices. For example, if you are downloading a file from the internet, your internet connection’s throughput will determine how long it takes to download the file.
Some factors can affect throughput, including:
- The type of connection: A wired connection typically has a higher throughput than a wireless connection.
- The distance between the two devices: The further apart the two devices are, the lower the throughput.
- The amount of traffic on the network: If there is a lot of traffic, the throughput will be lower.
You can improve throughput using a wired connection, reducing the distance between the two devices and network traffic.
Bandwidth
The bandwidth is the amount of data we can transfer per unit of time. For example, imagine we have a pipe with water running through it. If the pipe is large inside, more water can pass through. If the pipe is narrower, less water can pass through. In this case, the bandwidth is the maximum amount of water that can go through the pipe.
We measure bandwidth in bits per second (bps) in computer networks. The higher the bandwidth, the more data we can transfer per unit of time.
Bandwidth is essential for several reasons. For example, if you are streaming a video online, you need a high bandwidth connection to avoid buffering. Similarly, if you are downloading a large file, you will need a high bandwidth connection to complete the download quickly.
Several factors can affect bandwidth, including the type of connection you have, the amount of traffic on the network, and the distance between you and the server you are connecting to.
If you are experiencing problems with slow internet speeds, you can do several things to try to improve your bandwidth. For example, you can use a different type of connection, such as a cable modem or a fiber optic connection. You can also try connecting to a different server, or you can reduce the amount of traffic on your network.
Bandwidth is necessary to understand and distinguish throughput to design your system.
Analogy Explanation from Throughput, Latency, and Bandwidth
Now that we understand throughput, latency, and bandwidth, let’s see an analogy from those terms in a pipe and what they represent in the context of networks.
In the pipe, the water is the data, the pipe distance represents the latency, the thickness of the pipe is the bandwidth, and the water that goes through the pipe is the throughput:
Throughput vs. Bandwidth
Bandwidth and throughput are two terms that are often used interchangeably, but they actually have different meanings.
Bandwidth is the maximum data transfer rate that a network can support. It is measured in bits per second (bps). Throughput, on the other hand, is the actual data transfer rate that is sent over a network. It is also measured in bits per second (bps).
Bandwidth is determined by the physical characteristics of the network, such as the type of cable used and the number of nodes on the network. Throughput, on the other hand, is affected by a number of factors, including the amount of traffic on the network, the type of applications that are being used, and the distance between the sender and receiver.
It is important to understand the difference between bandwidth and throughput because it can help you to choose the right network for your needs. If you need to transfer large amounts of data quickly, then you will need a network with a high bandwidth. However, if you are only transferring small amounts of data, then a network with a lower bandwidth may be sufficient.
Here is a table that summarizes the key differences between bandwidth and throughput:
| Feature | Bandwidth | Throughput |
|---|---|---|
| Definition | Maximum data transfer rate | Actual data transfer rate |
| Unit of measurement | Bits per second (bps) | Bits per second (bps) |
| Determinants | Physical characteristics of the network | Amount of traffic, applications, distance between sender and receiver |
| Importance | Important for choosing the right network | Important for understanding network performance |
Conclusion
To understand throughput, latency and bandwidth is very important when designing a system in the cloud. In AWS or any other cloud vendor we will see those terms all the time. When working with Microservices, those terms will show up very often and not knowing what are those terms will make things far more difficult to understand a tool such as AWS or monitoring tools such as Datadog or Dynatrace.
