Networking

It’s a dangerous business, Frodo, going out your door. Kubernetes and public cloud infrastructure introduce a few layers of abstraction between users and our services. This article unravels some of those layers to help understand what, exactly, happens between the time a user makes a request to a Kubernetes service running in AWS and when the user receives a response. It’s helpful to start by framing a request in terms of the network boundaries involved, so let’s start there. ...

Before the Internet became a global network connecting millions of devices, it was a simple research experiment connecting a handful of institutions. In the beginning, the number of unique internet addresses could be measured in the tens. As the network expanded that number quickly grew into the hundreds and thousands and it became difficult to remember and type in IP addresses for each of these hosts. To manage the growing number of network hosts, a simple text file, called HOSTS.txt recorded each host and their IP address. To add your name to the hosts file, you needed to send an e-mail describing the changes you wanted to apply. The authority for the HOSTS.txt file would apply these changes once or twice a week and anyone who wanted to grab the updated list would periodically FTP to the canonical source, grab the latest file, and update their own list of hosts. Naturally, as this small network expanded into, and was eventually replaced by, the Internet, this solution became untenable – there were just too many hosts to keep track of, keep consistent, and to serve from a single canonical file using FTP and manual updates. HOSTS.txt did not scale. ...

To deliver real-time communication (RTC) from browser to browser requires a lot of technologies that work well together: audio and video processing, application and networking APIs, and additional network protocols that for real-time streaming. The end result is WebRTC — over a dozen different standards for the application protocols and browser APIs that enable real-time communication for the web. ...

There is no one-size-fits-all cellular network used across the world, and trying to understand how cellular technology works across all the different uses cases is difficult, if not impossible, in a short blog post. So, rather than trying to understand every possible standard, this article will focus solely on LTE networks. Fortunately, competing standards and implementations are roughly similar and we can extrapolate any lessons learned about LTE to other cellular networks without much difficulty. ...

WiFi, technically specified in the IEEE 802.11 set of standards, is one of the most widely deployed wireless standards in the world. Chance are the device you are using to read this article has is WiFi enabled. WiFi is a straightforward extension of Ethernet, with some slight adaptations for using radio instead of copper wire as the communication channel. Like Ethernet, WiFi has no central process that controls which device is allowed to transmit data at any point in time. Instead, each device decides on its own, and all devices must work together to guarantee good shared channel performance. With Ethernet, the protocol senses when the communication channel is busy and waits until it is free before sending data (carrier sensing). Ethernet also adds a collision detection protocol as an optimization. If a collision is detected, nodes stop sending data and use a randomized backoff algorithm to decide when to begin sending again. Together, these algorithms form the carrier-sense multiple access with collision detection algorithm (CSMA/CD). WiFi uses a similar strategy to Ethernet, but is restricted by the communication channel — there is no reliable way to detect collisions using radio waves. Therefore, instead of collision detection, WiFi uses a collision avoidance strategy defined by the carrier-sense multiple access with collision avoidance algorithm (CSMA/CA). ...

Ethernet is a technology developed a Xerox PARC in 1973 and 1974 to support local area networking. It has since been expanded to include a whole family of technologies that support various network architectures and topologies as part of the IEEE 802.3 working group dedicated to supporting networking using physical connections and devices (i.e. not wireless). Ethernet was built with the assumption that all computers on a network use a shared communication channel. The primary problem with using a shared connection is creating a sense of orderly traffic when an entire network of computers wants to constantly transmit data over the same channel. The core of that solution is sensing other node’s activity and waiting until the line is free and using collision detection to minimize the amount of traffic on the network. ...

All wireless networks (Wi-Fi, Bluetooth, 3G, LTE, etc.) operate using radio signals. Because they operate over the radio, all communication methods have a maximum channel capacity, regardless of technology. This maximum capacity was which is determined by the same underlying principles of information theory developed by Claude Shannon during World War II, known as Shannon-Hartley theorem or Shannon’s Law. Shannon’s Law states that This capacity relationship can be stated as: $$ {C=W\log _{2}\left( 1+{S \over N} \right)} $$ ...

The User Datagram Protocol (UDP), first described in 1980 by RFC 768, offers a minimal set of functionality: there is no guarantee of message delivery, no guarantee of message order, no congestion avoidance, and no tracking of connection state. In fact, UDP is often referred to as the null protocol, because it offers little functionality not already included in the IP layer. ...

The transmission control protocol (TCP) does one job very well — it creates an abstraction that makes an unreliable channel look like a reliable network. For applications built over an unreliable network like the Internet, TCP is a godsend that hides a lot of the inherent complexity in building networked applications. A laundry list of TCP features that application developers rely on every day includes: retransmission of lost data, in-order data delivery, data integrity, and congestion control. This article provides an introduction TCP, describing the structure of TCP segments, how TCP connections are established, and the algorithms that govern the flow of data between senders and receivers. ...

A WebSocket is a persistent connection between a client and server. WebSockets provide a bidirectional, full-duplex communications channel that operates over HTTP through a single TCP/IP socket connection. At its core, the WebSocket protocol facilitates message passing between a client and server. This article provides an introduction to the WebSocket protocol, including what problem WebSockets solve, and an overview of how WebSockets are described at the protocol level. ...