My favorite kind of blog post is when someone takes a subject that I’ve spent all of five minutes considering and then says—no!—this is an enormous topic worthy of a dissertation. Look at all the things you can do with this tiny CSS property!
I was reminded of this when I spotted this post by Josh Comeau about designing beautiful shadows in CSS:
In my humble opinion, the best websites and web applications have a tangible “real” quality to them. There are lots of factors involved to achieve this quality, but shadows are a critical ingredient.
When I look around the web, though, it’s clear that most shadows aren’t as rich as they could be. The web is covered in fuzzy grey boxes that don’t really look much like shadows.
Josh shows the regular old boring shadow approaches and then explores all the ways to improve and optimize them into shadows with real depth. It all comes down to taking a closer look color and exploring the box-shadow CSS property. And speaking of depth, Rob O’Leary’s “Getting Deep Into Shadows” is another comprehensive look at shadows.
I had also completely forgotten about filter: drop-shadow; which is particularly useful on adding shadows to images that you want to throw onto a page. Great stuff all round.
This article is actually an excerpt from Wladston Ferreira Filho‘s new book Computer Science Unleashed. This book is about all the groundbreaking technologies behind the World Wide Web. We might even take them for granted these days, but there are important and learnable technologies behind how it all works. Read on and marvel at the engineering ingenuity that enables simple physical links between computers to become a global, near-instant communication medium that everyone can use almost for free.
Humans crave connections, and the advent of the digital revolution has empowered us to be more connected than ever before. The Internet has unleashed upon billions of people unprecedented economic and political freedom, as well as powerful means of control and domination. Yet, the vast majority of us are oblivious to its inner workings.
Skilled people who can program computers to use the Internet are at the vanguard of the digital revolution. This chapter will teach you how the Internet works, so you can join this select group. You’ll learn to:
Link computers to one another to make a network,
Combine networks using the Internet Protocol,
Locate a recipient from its Internet address,
Find a route through the Internet to that location,
Transport data between distant applications.
Before the Internet, telecommunication between two parties required a direct physical link. In the 1950s, each telephone had a wire leading directly to a central station. For a call to go through, an operator had to physically connect the wires of two telephones. For long distance calls, wires were laid out between distant stations, and several operators in different places had to physically connect the chain of wires linking the two phones.
The Internet did away with this. Wires aren’t physically reconfigured to create direct, exclusive links. Instead, the information is retransmitted step by step via a chain of linked devices until it reaches its destination. This eliminates the need for wire operators and central coordination. Also, wires are no longer constrained to serve a single connection–many concurrent connections can share the same wire. This allows global communications to be instant, cheap and accessible.
However, modern networking technology is more intricate than early telephony. It has many successive layers, each building on top of the previous. Let’s explore how connections are made at these different levels, starting with the most basic layer.
1.1 Links
A direct connection between two computers is achieved through a transmission medium: a physical channel where signals flow. It can be a copper wire carrying electric currents, a fiber-optic cable directing light, or air hosting radio waves. Each connected computer has a network interface to send and receive signals in the transmission medium. For instance, cellphones have a radio chip and antenna to handle radio signals traveling through the air.
Figure 1.1 A link is established between two network interfaces if they share a transmission medium and agree on the rules of communication.
In order to communicate, network interfaces must agree on the rules to follow when sending and receiving signals. This set of rules is called the link layer.
When a medium exclusively connects two computers, we say they maintain a point-to-point connection, and their link layer relies on the most basic set of rules: the Point-to-Point-Protocol (PPP). It merely ensures the two computers can identify each other and exchange data accurately.
However, connected computers don’t always get to enjoy such an exclusive link. Often, they must share the transmission medium with several other computers.
Shared Links
One way to link computers in an office is to plug each of them into a hub with a wire. The hub physically connects all the wires that reach it, so a signal sent by one computer will be detected by all the others! This will also happen on your home WiFi, since the same radio frequency is used by all connected devices. Communications can become messy if all of them use the medium at the same time.
Figure 1.2 A message sent on a shared link will be detected by all.
The link layer contains a set of rules to define how computers should share their communication medium, fittingly called Medium Access Control (MAC). The rules resolve two main challenges:
COLLISIONS — If two computers send a signal through the same medium at the same time, the resulting interference garbles both transmissions. Such events are called Collisions. A similar problem occurs when your group of friends or family talk over each other hand no single voice can be clearly heard.
There are methods to avoid collisions. First, only start transmitting signals when no other computer is transmitting. Second, monitor your communications–if a collision occurs, wait for a brief but random amount of time before trying to transmit again.
Figure 1.3 Collision between Ada and Andrew.Figure 1.4 Ada and Andrew both resend after a random duration.
These methods have some limitations. If there are too many transmission attempts through a medium, collisions will occur relentlessly. We say the link is saturated when excessive collisions break down communications. Have you ever been frustrated at a large venue because your phone wouldn’t send text messages or make calls? This may happen if too many phones are attempting to communicate concurrently and the cellular link becomes saturated.
PHYSICAL ADDRESSING — Ada and Charles have a direct link between their computers. Ada wants to talk with Charles, so she transmits a signal with her message through the medium. However, the medium is shared, so everyone linked to the medium gets the message. How can the other computers know that the signal they picked up was not destined for them?
Figure 1.5 Andrew’s network interface discards the message.
Each computer’s network interface has an identifier, known as its physical address or hardware address. A transmission in a shared medium must begin with two such addresses: that of the recipient and that of the sender. Upon receiving a transmission, a computer will know if it should be ignored or picked up and to which address it should reply.
This can only work if physical addresses are unique: if two computers use “my_netinterface”, we’re back to square one. For this reason, virtually all network interfaces follow a naming scheme defined in the rules of Medium Access Control. These standard physical addresses are called MAC addresses.
MAC Addressing
Computers, smartphones, smart watches, and smart televisions can each have WiFi, Bluetooth, and Ethernet network interfaces. Each network interface has its own, unique MAC address marked into the hardware during production. You should not worry about assigning a MAC address to your computer: you can always use the one that came with its network interface.
Since MAC addresses are simply large random-looking numbers, network interface manufacturers around the world must coordinate to avoid accidentally assigning the same number to two different devices. To this end, they rely on the Institute of Electrical and Electronics Engineers (IEEE), that assigns each of them a different range of MAC addresses.
A MAC address is expressed as six pairs of hexadecimals1 separated by colons. The first half of the address is an identifier assigned by the IEEE to a unique manufacturer. This manufacturer then chooses a unique second half for each network interface.
60:8B:0E:C0:62:DE
Here, 608B0E is the manufacturer number. This specific number was assigned by IEEE to Apple, so this MAC address should belong to an Apple device.2 A device’s MAC address is often written on a label stuck to the packaging or on the device itself, next to the serial number.
Figure 1.6 Each MAC address is unique.
There’s a special address reserved for transmissions to all computers in a medium. It’s called the broadcast address, and it reads FF:FF:FF:FF:FF. You use it when you try to connect to an unknown device. For instance, when your smartphone’s WiFi card isn’t deactivated, it persistently broadcasts to FF:FF:FF:FF:FF that it’s looking for an access point. Discoverable access points will respond with their own MAC address so you can establish a link.
Such discovery broadcasts, like all other transmissions, contain the sender’s MAC address. Walking around with a smartphone can therefore be like walking around with a loudspeaker shouting your name non-stop, only using radio waves instead of sound and the MAC address instead of your moniker. In 2013, Edward Snowden revealed that the NSA3 monitored the movements of people by sniffing WiFi transmissions in big cities, storing records of where each MAC address was seen.
You can also set your own network interface to promiscuous mode, and it will pick up all transmissions regardless of their intended recipient. Doing so allows you to discover hidden WiFi networks, to list which MAC addresses are in your area, and sometimes even to read the contents of other people’s transmissions. Browsing the Internet through an unsecured WiFi network can therefore be unsafe: your communication is broadcast for anyone in range to hear. This is why encryption4 is important for WiFi’s link layer.
Be careful: a network interface can be configured for its transmissions to start with any MAC address for both the recipient and the sender. Nothing stops a malicious agent from impersonating you by using your MAC address in their transmissions. This type of attack is known as MAC spoofing. When the link layer was invented, security wasn’t a concern. Protocols are evolving to become more secure and neutralize such attacks, but it’s an ongoing process.
Frames
Sometimes, a transmission must contain a lot of data, and sending out a single, big fat message is impractical. Network interfaces and computers are not all capable of the same transmission speeds. Moreover, what would happen if a collision occurred in the middle of the transmission? The entire transmission would have to be discarded, as it would be difficult for the sender and receiver to determine exactly which parts of the message were received and which were not.
To solve these issues, long messages are always split into small parts, each sent as an independent transmission. The duration between transmissions can vary according to the capabilities of both computers: slower devices needs longer breaks. If an error occurs, it is only necessary to discard and resend the small transmission that failed.
Figure 1.7 An Ethernet frame. Once it is transmitted in a copper wire, it becomes a series of electric signals that encode a number. The Ethernet protocol instructs how to interpret this number. For instance, the first 12 hex digits of the number encode the destination MAC address.
Each independent transmission is called a frame. Standard WiFi protocols cap the size of frames to 2,346 bytes. Thirty-four bytes are needed for MAC addresses and error-detecting codes. Therefore, a WiFi frame can ultimately carry up to 2,312 bytes of data, called the payload.5 In wired networks, the maximum frame size is usually 1,526 bytes, with room for a 1,500 byte payload.
On rare occasions, disturbances in the medium interfere with a transmission, and the receiver picks up signals that don’t encode exactly the same information that the sender intended to transmit. Let’s see the special field that was added to address this problem.
FCS — The last part of the frame is the FCS (Frame Check Sequence), and it ensures that information was transmitted accurately. It doesn’t add new information to the transmission: it is merely the result of a calculation using the contents of all other fields. Changing any content before the FCS should cause the FCS number to change as well.
Upon receiving a frame, a computer calculates the expected FCS number from the information it received and compares it to the received FCS. If they don’t match, the frame is discarded. If they match, we know that the message wasn’t garbled and trust that the received payload is error-free.
TYPE — The frame shown in Figure 1.7 has one last field we haven’t talked about: the payload type. It tells the receiver which rules should be followed to interpret the data in the frame’s payload. In the next section, we’ll explore the most common set of such rules.
1.2 Internet
We’ve seen that the link layer enables directly connected computers to exchange messages inside frames. The internet layer, also known as the network layer, specifies how to transmit these messages between computers that are not directly connected.
The trick is to equip some computers, called routers, with multiple network interfaces. All computers in a network are then linked to at least one router, and all routers are linked to at least one other router. When a router receives a message at one of its network interfaces, it can forward it to another router through a different network interface.
LOCAL AREA NETWORKS — We can ask a router we’re linked with to forward a message to a computer we’re not linked with. Suppose you have a wired network in your home connecting a router and a desktop computer. Suppose the router is also directly connected to a smartphone in a different, wireless network.
Even though the desktop computer and the smartphone are not directly connected to the same network, they can send messages to each other using the router as a relay. Computers from different networks in close vicinity that can talk to each other through routers form a larger network, called a Local Area Network (LAN).
In a home or small office, one router will be enough to link all the computer networks in the area. When assembling a LAN that covers a large organization such as a university or hospital, many routers may be required to link all the different computers networks into a fully connected system.
Figure 1.8 In this small LAN, Ada and Andrew can send messages to each other through their router Charles.
WIDE AREA NETWORKS — But why stop there? If your router is linked with a router outside your home, which in turn is linked with a router at the university, you can ask for your message to be forwarded to computers on the university’s LAN. When distant LANs are connected to each other, they form a Wide Area Network (WAN).
Figure 1.9 Charles is connected to a distant router, Marie, and they both forward messages around this WAN.
A WAN can grow larger as more LANs are connected to it. Different WANs can also be connected to form an even larger WAN. The largest WAN in the world is a collection of thousands of interconnected networks that we call the Internet. It’s the network we use every day to send emails and browse the web. In 2019, this WAN contained over a billion computers. Let’s see how they all got connected.
Interconnection
The most straightforward way to connect your router to the Internet is to pay for it. Some organizations on the Internet will link one of their routers to yours, and allow messages to and from your network to pass through their network via this link. This paid service is called transit, as all of your messages will transit through their network before going to the specific router you’re aiming for.
However, transiting through a third party network is not always necessary in order to connect to another router of the Internet. If, for example, two nearby universities communicate a lot, they can link their routers in order for messages to flow directly between their networks. This can save money, as these messages would otherwise have to transit through a paid connection. The free exchange of messages between the networks of different organizations is called peering.
Routing
Any computer linked to a router of the Internet can ask for its messages to be forwarded by other routers. Messages can be routed over large distances. For instance, there is a system of submarine cables linking routers in many coastal cities:
Figure 1.10 The SAm-1 system links routers in 16 cities from 11 different countries, using over 15 thousand miles of underwater cables.
There is no direct link between the routers in Miami and Buenos Aires. However, Miami is linked with Puerto Rico, which is linked with Fortaleza, which is linked with Rio de Janeiro, which is linked with Buenos Aires. Miami and Buenos Aires can exchange messages through these cables if routers along the way forward the messages back and forth. Today, there are submarine cables linking hundreds of coastal city routers around the globe:
Figure 1.11 Fiber-optic submarine cables currently in service.
Virtually every other city on Earth is directly or indirectly linked to these coastal cities, often through cables in the ground. Communication satellites also have routers to establish wireless links to remote locations. All routers can forward messages, so a message you send on the Internet can be routed to any other computer on the Internet. That is, if a path to it can be found.
Location Addressing
In the link layer, computers are identified by a physical address. Physical addresses uniquely identify computers, but they don’t give any hints on where a computer is connected and how it can be reached. If the computer moves to the other side of the world, it will retain its physical address!
Suppose you mailed a package to Louis through the post along with a picture of him instead of his address. This package has a defined destination; however, an international postal service would have no way of knowing which direction the package should be sent in order to deliver it to Louis.
Post offices must first know to which country the package should go. The first post office in that country should then know to which province or state it should go. The next post office should know the city, and the final post office, the street address. An address containing all this information is called a hierarchical address. As with post offices, routers need a hierarchical address of the package recipient’s location:
Figure 1.12 Ada wishes to send a package to Louis, so she requests her router Charles to forward it. She writes on the package a hierarchical address of Louis. Charles then knows he must send the package to France, so he sends it to the French router he is linked with: Marie.
For this mechanism to work on a global scale, all computers involved must follow the same set of rules to create and handle package forwarding requests. A computer in China must understand a request from a computer in Nigeria, even though the two may use different languages, operating systems and hardware.
Figure 1.13 “Before the Internet”, courtesy of http://xkcd.com.
Internet Protocol
We’ve seen a computer must follow the rules of Medium Access Control to establish a link with another computer. Similarly, it must follow the Internet Protocol, or IP,6 to ask routers to forward messages to other computers on your LAN or on the Internet.
A message forwarding request that follows the IP rules is called an IP packet. The IP packet is essentially a big number, where digits in specific positions encode key information. Virtually all computers understand IP packets and are able to forward them. This makes an IP packet easily movable from one computer to the next, until it reaches its destination.
An IP packet contains the location addresses of its sender and recipient, followed by whatever data they want. To send an IP packet, we transmit a frame where the payload is the IP packet, and the frame type is 86DD. When a router receives a frame of this type, the IP packet is re-transmitted in another frame to the next computer in the path of the packet’s destination.
Figure 1.14 Ada sends an Ethernet frame to her router Charles containing an IP packet for Louis. The Ethernet frame therefore contains the physical address of Charles and the packet contains the location address of Louis. Charles will then forward the packet inside a new frame of his own containing the physical address of someone in France.
In order for IP packets to be forwarded around universally, everybody must agree on a standard for location addressing. We’ve seen how physical addresses are allocated by manufacturers according to the rules of Medium Access Control. Let’s now learn how the Internet Protocol does this for location addresses. We will then see how the Internet Protocol defines routing rules based on these addresses.
1.3 IP Addressing
The Internet Protocol sets the rules on how location addresses work–that’s why they’re called IP addresses. Computers can only send or receive IP packets after they get an IP address. Permission to use a group of IP addresses is first granted to an organization. These addresses are then assigned to computers which are directly or indirectly associated with the organization.
In order to explain how this process works, let’s define what IP addresses are and how they’re written.7 An IP address is a number 128 bits long.8 They’re typically written in hex, with colons separating eight groups of four digits. This is Facebook server’s IP address:
2a03:2880:f003:0c07:face:b00c:0000:0002
IP addresses can be shortened by omitting the leading zeros of any four-digit block:
2a03:2880:f003:c07:face:b00c::2
As with a postal address with country, city and street, IP addresses are hierarchical for routing to be possible. While the broadest part of a postal address is the country, the broadest part of an IP address is the routing prefix (2a03:2880).
The prefix shows up as the first digits of an IP address. Once an organization is granted such a prefix, it has the right to assign any IP address that begins with that prefix to its computers. The prefix has a variable length: organizations that have more computers to manage are granted shorter prefixes. Some organizations are even granted multiple prefixes.
For example, we know that all addresses that begin with 2a03:2880 are assigned to computers inside Facebook’s network. Those that begin with 2c0f:fb50:4002 are in Google’s network in Kenya. For its data center in Singapore, Google was granted the prefix 2404:6800.
For routing purposes, the LANs and WANs that share the same prefix are organized in small networks called subnets. The digits after the routing prefix and up to the middle of an IP address indicate in which subnet (f003:c07) a computer can be found.
2a03:2880:f003:c07:face:b00c::2
This means there’s a network at Facebook where all computers have IP addresses that begin with 2a03:2880:f003:c07. Together, the routing prefix and the subnet form the network ID (2a03:2880:f003:c07) of an IP address. The network ID is always 16 digits long (including omitted zeros). This means an organization with a longer routing prefix can have less subnets within it.
Finally, the next 16 digits of an IP address are called the interface ID (face:b00c::2), as they identify a specific network interface within a subnet. Many network administrators simply fill in this part of the IP address with the device’s MAC address. These digits can be any number, as long as it’s only used once per subnet.
For this addressing system to work universally, there must be a mechanism to ensure no two organizations use the same routing prefix. As was the case for MAC addresses, engineers solved this through some international coordination.
Figure 1.15 Don’t ask your boss where she lives!
IANA
Engineers worldwide agreed that an American non-profit organization, the Internet Assigned Numbers Authority (IANA), decides who gets control over which IP routing prefixes. In practice, IANA delegates most of its power to five non-profit organizations called Regional Internet Registries, or RIRs. To do so, it allocates each RIR short hex combinations that they can use as the first digits of the routing prefixes they assign.
Figure 1.16 Examples of allocations to each RIR.Figure 1.17 IANA delegates its IP addressing power geographically: each RIR is responsible for a different region.
To obtain a routing prefix for your organization, you must make a request to the RIR of the region where your routers will be. That RIR will then assign you a prefix starting with one of their combinations of hex digits that IANA allocated them.
For example, Facebook, which has headquarters in Ireland, was granted its routing prefix by RIPE NCC. Likewise, the Swiss bank Credit Suisse has a Latin American branch that was granted a routing prefix by LACNIC:
Figure 1.18 IP address allocation chain for two companies.
This means computers in the Latin American Credit Suisse branches may be assigned IP addresses as follows:
2801:80:1380: ■ ■ ■ ■ :____:____:____:____
Network administrators in the bank will assign a unique combination of hex digits to each of their subnets such that they fit in the remaining space of the network part . Since each hex digit can have 16 different values, the bank has enough space for 164= 65,536 different subnets. Facebook, being a larger organization, was granted a prefix with room for over 4 billion subnets!
We’ve seen that network administrators can choose how the sixteen blanks of the interface ID are to be filled for individual devices. Such devices may then send and receive IP packets to and from the Internet as long as their router has connectivity.
Internet Service Providers
Most individuals and small organizations don’t deal directly with RIRs, nor do they maintain peering links to other computer networks. Instead, they buy Internet connectivity from specialized companies, which are called Internet Service Providers (ISP). ISPs install routers close to their customers. That way, they can easily link one of their routers to a router in any customer’s premises. They also allocate a routing prefix for each of their customers.
Let’s see how it works in practice. In the United Kingdom, an ISP called Sky was granted the routing prefix 2a02:0c7f. Sky operates in many British cities, so the prefix is divided between their regional bases. For instance, they assign 2a02:c7f:48to their Milton Keynes network and 2a02:c7f:7eto the one in Romford.9
Let’s suppose Ada lives in Romford and wants to set up a network in her home. She has a desktop computer and a printer which she wants to connect using an Ethernet wire. She also wants her own WiFi network to connect her smartphone, tablet and laptop.
Ada hires Sky, and they link their Romford router to a router in her home. Sky assigns Ada’s router a 14-digit routing prefix based on the one of their Romford base. Each network in Ada’s home (wired and wireless) gets assigned a subnet, based on the routing prefix Sky allocated to Ada. Figure 1.19 on the next page shows the full IP address allocation path from IANA to each of Ada’s devices.
Ada’s router receives IP packets from several different computers, yet it’s easy for her router to decide on which link to forward each packet it receives. Packets addressed to a computer in one of Ada’s subnets can be directly delivered. All other IP packets it receives are forwarded through the link to the ISP.
Figure 1.19 IP address allocations from IANA to Ada’s devices. Her router uses different subnets for her wireless and wired networks, and therefore has a different IP address for each.
For routers that don’t rely on an ISP, it’s not so easy: they obtain connectivity from links with several routers from multiple computer networks. But how do they decide on which link they should forward an IP packet? And how can they be sure that they are forwarding it to a router closer to their final destination?
1.4 IP Routing
Suppose Ada wants to send a message to Facebook from her laptop. She will use the Internet Protocol, so she starts by crafting an IP packet that includes her own IP address, Facebook’s IP address, and her message as the payload. She then transmits the packet in a WiFi frame from her laptop to her home router:
Several routers, starting with the one at Ada’s home, retransmit the packet until it reaches Facebook. Along the way, each of those routers must choose in which direction the packet should “hop” to reach the next router. The last router will then make the packet “hop” towards its final destination computer.
Tables of Addresses
Routers choose the next hop of a packet based on its destination IP address. In order to do so, they are equipped with a table filled with addresses. Rows list possible IP addresses the router is configured to recognize. For each address, the table indicates which computer should be the next hop of a packet destined to that address. Every router has a unique table that reflects how the router is linked. For example, here is how Ada’s router is linked:
Figure 1.21 Ada’s router is connected to a desktop computer and a printer via Ethernet, a notebook and a smartphone via WiFi, and to the ISP via DSL (or Digital Subscriber Line, a technology that allows digital data to flow through old telephone cables).Figure 1.22 Table that guides Ada’s router to correctly forward IP packets to computers shown in Figure 1.21.
If the router receives a packet whose destination IP address doesn’t match any row in the table, the packet is forwarded through to the default route. For Ada’s router, routing is simple: a packet is either directly delivered to a computer in her home or forwarded to Sky Romford, her ISP.
Routing is more complicated for the ISP’s router. In addition to its peering and transit links, it receives packets from many different customers. For simplicity, let’s suppose Sky Romford’s router only serves two customers and has two peering links: one to the Sky router in Milton Keynes, and the other to Oxford University. Finally, let’s imagine it has a transit link with a larger telecom company:
Figure 1.23 Map of Sky Romford links. Ada, Charles, and Oxford University can talk using Sky’s local infrastructure only.Figure 1.24 Table that guides Sky Romford’s router to correctly forward IP packets to networks shown in Figure 1.23.
This is where the IP addressing hierarchy comes in handy. In the forwarding table of fig. 1.24, IP addresses are grouped according to their routing prefix. This works because all IP addresses starting with 2a0a:207 are from computers in Oxford University, and all IP addresses starting with 2a02:c7f:48are from computers serviced by Sky in Milton Keynes.
Internet Exchange Points
In order to increase capacity and speed, network administrators often set up peering links with as many other organizations as possible. The cheapest way to do this is through places called Internet Exchange Points, or IXPs. Organizations join an IXP by wiring their routers to the IXP building. Every participating organization can then establish individual peering links with other organizations connected to the building.11
In fig. 1.23, only two peering links were shown for clarity’s sake. A typical ISP actually has scores of peering links per IXP they’re wired to. In addition, it’s common in big cities for Internet corporations like Netflix and Google to establish peering links directly with ISPs, allowing them shorter and faster connections to many of their customers.
Internet Backbone
ISPs and other telecom companies typically expand their interconnections as much as possible by establishing peering links wherever they can. However, in order to reach networks they cannot peer with, they have to buy transit from other operators.
There is a handful of companies in the world that don’t pay anyone for transit. These companies operate huge networks that all peer with each other, allowing regional ISPs to be interconnected globally. These huge networks are called Tier-1 networks, and they form the backbone of the Internet. Some Tier-1 networks are operated by AT&T, Verizon, and Lumen.12
Dynamic Routing
Large telecom companies must maintain connectivity even if some of their transit or peering links break down. This means that they can’t rely on a single link for each routing prefix in their table of addresses. In fact, they have dynamic routers that map out how other networks are interconnected in order to choose which routes to prioritize in their tables.
Dynamic routers periodically exchange information with other dynamic routers they’re linked to. They tell each other which network prefixes are reachable through each of their links. This allows them to determine how many hops away each link is from every routing prefix and where these hops occur. Dynamic routers can then determine the best route to each prefix based on metrics like distance and speed.13
With this information, dynamic routers build a table that covers all routing prefixes. For each prefix, the table indicates which next hop is on the best route to the final destination. When a link is established or a link goes down, dynamic routers inform their peers. As the news spreads, all of them update their tables to keep forwarding packets towards the best routes.
There is no central entity coordinating the exchange of this information: routers share link details with their peers freely and voluntarily. Consequently, routing problems often emerge.
Routing Loop
Misconfigured routers can provoke errors. Most notably, bugged tables of addresses can send a packet back a few hops, and it gets caught in an endless cycle of doom:
Figure 1.25 Bugged tables sending a packet round in circles.
If the tables aren’t corrected, more packets with the same intended destination will be endlessly forwarded in circles. Too many packets can even saturate and clog the links. This is known as a routing loop problem. Fortunately, the Internet Protocol provides a way to identify the issue when it occurs.
HOP LIMIT — To interrupt perpetual routing loops, all IP packets carry a hop limit between 0 and 255. It indicates the number of times the packet can be forwarded by routers. Typically, packets are created with a hop limit of 64. Whenever a router forwards a packet, it reduces the hop limit by one:
Figure 1.26 A packet’s hop limit is the only element of an IP packet that routers change while forwarding.
If a packet is going around in circles, its hop limit will eventually reach zero. If a router receives an IP packet with a hop limit of zero, it can be discarded. An IP packet containing an error message should then be transmitted back to the sender by the last router, stating that the packet could not be delivered because its hop limit was reached.
Feedback through such error messages helps network administrators fix critical bugs, and routing loops are not the only ones. In fact, the Internet Protocol covers how a variety of routing problems should be dealt with.
Diagnostics
Routers discard IP packets that they are unable to handle. When this happens, they send an informational message about the incident to the packet’s sender. The Internet Protocol defines how routers must format such messages, ensuring they can be understood by any computer. These rules are a subset of the Internet Protocol called the Internet Control Message Protocol (ICMP).
ICMP assigns error codes to the most common routing problems. To report a problem, a router sends an IP packet containing the error code as the message body, formatted according to ICMP rules. Let’s see some common problems that can be reported using ICMP, starting with the routing loop problem.
TIME EXCEEDED — If a router receives an IP packet with a hop limit of zero, its travel time is up. The packet either got stuck on a routing loop, or it was granted an insufficient hop limit by the sender.
In such cases, an ICMP message with a time exceeded error code is sent back. The ICMP message includes the first bytes of the discarded packet to allow the original sender to know which packet didn’t make it to its destination.
Figure 1.27 Once a router receives a packet with a hop limit of zero, it is discarded and an ICMP error message is sent to the packet’s sender.
Notice that the IP packet Charles sends back in fig. 1.27 includes a protocol field. It’s a two-digit hex number identifying how the packet’s payload should be interpreted. The latest version of ICMP was assigned the protocol number 0x3A. All IP packets must include a protocol number. In the next section, we’ll learn more about this. For now, let’s explore other common routing problems.
DESTINATION UNREACHABLE — Sometimes, a router has nowhere to send a packet. This can happen for many different reasons, for example if the IP address isn’t in the router’s table of addresses, and the table doesn’t propose a default next hop. Sometimes, the next hop happens to be offline.
When the router doesn’t know where to forward the packet, it returns an ICMP message with the destination unreachable error code, along with the first bytes of the discarded packet’s content.
PACKET TOO BIG — We’ve seen that link layer protocols limit the amount of data that can be sent in a single frame. Frames from different types of network links can carry payloads of different sizes.
The maximum number of payload bytes that can be carried in a single frame is called its Maximum Transmission Unit (MTU). Different link layer protocols have different MTU values. For Ethernet frames, the MTU is 1,500. For WiFi frames, it’s 2,305.
If a router receives a larger packet than what the next hop can handle, it can’t be forwarded as it stands. Instead, the router returns an ICMP message with the packet too big error code, the first bytes of the problematic packet, and the MTU of the next hop. The informed sender can then trim or split the original message into smaller packets before trying again.
PARAMETER PROBLEM — An IP packet contains a lot of extra information alongside its payload. We’ve seen it contains IP addresses, a hop limit, and a protocol number. It also includes a field indicating the size of the payload and another specifying the version of the Internet Protocol it respects. Additional fields are also there to help routers prioritize important packets.
All these fields must be ordered and formatted according to strict rules. When a router receives a packet that doesn’t conform to the protocol, it returns an ICMP message with the parameter problem error code and the location in the packet where the conflict was found. As usual, the ICMP message also contains a few bytes of the discarded packet for identification purposes.
INFORMATIONAL MESSAGES — Error reports are not the only messages ICMP defines to inspect and diagnose faulty computer networks. Most notably, the echo request and echo reply informational message pair is widely utilized. When a computer receives an ICMP echo request, it returns a packet containing an ICMP echo reply.
This is useful to test if a computer is online. There’s a program called ping that sends out an ICMP echo request message and measures how long it takes for the reply to reach you.14 Furthermore, by sending ICMP echo requests with different initial hop limits, you can trace the route packets follow to reach their destination.15
We’ve seen that computers on the Internet can exchange information—such as ICMP messages—in IP packet payloads. However, the true power of the Internet is unleashed when applications, not computers, start using IP packet payloads to send each other data. This requires extra information to be included in the IP packets so that a computer can handle multiple streams of data for the different applications it runs. This extra information is described by the transport layer, which includes the famous TCP and UDP protocols.
To keep learning about these protocols, check out our book, Computer Science Unleashed. The book will also teach you how famous Internet applications such as email and the Web works. And it explains how DNS and domain names function under the hood! Understanding these technologies is essential for being a well-rounded web developer. Besides the Internet, Computer Science Unleashed also covers other three groundbreaking technologies: data analysis, machine learning, and cryptography. And it also includes a bonus chapter explaining how to use regular expressions!
1In day-to-day life, we almost always express numbers in decimal form, where each digit is one of ten characters: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9. Computer scientists, on the other hand, like expressing numbers in hexadecimal form, where each digit can be one of sixteen characters: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f. For more about number bases, see Appendix I.
2You can look up who manufactured a device by entering the first six digits of its MAC address at http://code.energy/mac-lookup.
3National Security Agency, a US government spying organization.
4Encryption allows messages to look garbled to eavesdroppers.
5If we encode one byte per character, a WiFi frame has room for about 500 words, enough to fill a page of text.
6By “IP”, we mean its latest version, IPv6. A legacy version of the protocol, IPv4, is still used, despite being released in 1981. IPv4 can only support about 3 billion computers. IPv6, launched in 2012, can support a virtually unlimited number of computers. As of 2020, a third of the Internet’s computers use IPv6.
7We’ll present IP addresses as defined in the latest version of IP. Legacy IPv4 addresses are still used. They are written as four groups of up to three digit decimal numbers, separated by dots, for example, 192.168.0.1.
8It takes 128 zeros and ones to write the number. This means it’s a number between 0 and 340,282,366,920,938,463,463,374,607,431,768,211,456.
9This information is public, you can look up the network location of any routing prefix. The practice is called IP geolocation, and it’s how websites guess the country and city you browse from.
10We’ve included the fields of the WiFi frame which also exist in Ethernet frames. A WiFi frame has more fields, which were hidden for simplicity.
11IXPs are extremely important for making the Internet well connected and cheap. This video explains why: http://code.energy/IXP.
12For an idea of how colossal these networks are, Lumen alone manages and operates 750,000 miles of fiber optic cables. That’s more than enough cable to reach the moon, three times!
13All five RIRs constantly disclose information on all routing prefixes they delegate. Dynamic routers closely track these announcements, so they can ensure their tables have a row for every existing routing prefix.
There is a helluva gotcha with styling a <details> element, as documented here by Kitty Guiraudel. It’s obscure enough that you might never run into it, but if you do, I could see it being very confusing (it would confuse me, at least).
Perhaps you’re aware of the shadow DOM? It’s talked about a lot in terms of web components and comes up when thinking in terms of <svg> and <use>. But <details> has a shadow DOM too:
<details> #shadow-root (user-agent) <slot name="user-agent-custom-assign-slot" id="details-summary"> <!-- <summary> reveal --> </slot> <slot name="user-agent-default-slot" id="details-content"> <!-- <p> reveal --> </slot> <summary>System Requirements</summary> <p> Requires a computer running an operating system. The computer must have some memory and ideally some kind of long-term storage. An input device as well as some form of output device is recommended. </p> </details>
As Amelia explains, the <summary> is inserted in the first shadow root slot, while the rest of the content (called “light DOM”, or the <p> tag in our case) is inserted in the second slot.
The thing is, none of these slots or the shadow root are matched by the universal selector *, which only matches elements from the light DOM.
So the <slot> is kind of “in the way” there. That <p> is actually a child of the <slot>, in the end. It’s extra weird, because a selector like details > p will still select it just fine. Presumably, that selector gets resolved in the light DOM and then continues to work after it gets slotted in.
But if you tell a property to inherit, things break down. If you did something like…
<div> <p></p> </div>
div { border-radius: 8px; } div p { border-radius: inherit; }
That <p> is going to be square as a square doorknob. I guess that’s either because you can’t force inheritance through the shadow DOM, or the inherit only happens from the parent which is a <slot>? Whatever the case, it doesn’t work.
Many developers love working with static site generators like Gatsby and Hugo. These powerful yet flexible systems help create beautiful websites using familiar tools like Markdown and React. Nearly every popular modern programming language has at least one actively developed, fully-featured static site generator.
Static site generators boast a number of advantages, including fast page loads. Quickly rendering web pages isn’t just a technical feat, it improves audience attraction, retention, and conversion. But as much as developers love these tools, marketers and other less technical end users may struggle with unfamiliar workflows and unclear processes.
The templates, easy automatic deploys, and convenient asset management provided by static site generators all free up developers to focus on creating more for their audiences to enjoy. However, while developers take the time to build and maintain static sites, it is the marketing teams that use them daily, creating and updating content. Unfortunately, many of the features that make static site generators awesome for developers make them frustrating to marketers.
Let’s explore some of the disadvantages of using a static site generator. Then, see how switching to a dynamic content management system (CMS) — especially one powered by a CRM (customer relationship management) platform — can make everyone happy, from developers to marketers to customers.
Static Site Generator Disadvantages
Developers and marketers typically thrive using different workflows. Marketers don’t usually want to learn Markdown just to write a blog post or update site copy — and they shouldn’t need to.
Frankly, it isn’t reasonable to expect marketers to learn complex systems for everyday tasks like embedding graphs or adjusting image sizes just to complete simple tasks. Marketers should have tools that make it easier to create and circulate content, not more complicated.
Developers tend to dedicate most of their first week on a project to setting up a development environment and getting their local and staging tooling up and running. When a development team decides that a static site generator is the right tool, they also commit to either configuring and maintaining local development environments for each member of the marketing team or providing a build server to preview changes.
Both approaches have major downsides. When marketers change the site, they want to see their updates instantly. They don’t want to commit their changes to a Git repository then wait for a CI/CD pipeline to rebuild and redeploy the site every time. Local tooling enabling instant updates tends to be CLI-based and therefore inaccessible for less technical users.
This does not have to devolve into a prototypical development-versus-marketing power struggle. A dynamic website created with a next-generation tool like HubSpot’s CMS Hub can make everyone happy.
A New Generation of Content Management Systems
One reason developers hold static site generators in such high regard is the deficiency of the systems they replaced. Content management systems of the past were notorious for slow performance, security flaws, and poor user experiences for both developers and content creators. However, some of today’s CMS platforms have learned from these mistakes and deficiencies and incorporated the best static site generator features while developing their own key advantages.
A modern, CMS-based website gives developers the control they need to build the features their users demand while saving implementation time. Meanwhile, marketing teams can create content with familiar, web-based, what-you-see-is-what-you-get tools that integrate directly with existing data and software.
For further advantages, consider a CRM-powered solution, like HubSpot’s CMS Hub. Directly tied to your customer data, a CRM-powered site builder allows you to create unique and highly personalized user experiences, while also giving you greater visibility into the customer journey.
Content Management Systems Can Solve for Developers
Modern content management systems like CMS Hub allow developers to build sites locally with the tools and frameworks they prefer, then easily deploy to them their online accounts. Once deployed, marketers can create and edit content using drag-and-drop and visual design tools within the guardrails set by the developers. This gives both teams the flexibility they need and streamlines workflows.
Solutions like CMS Hub also replace the need for unreliable plugins with powerful serverless functions. Serverless functions, which are written in JavaScript and use the NodeJS runtime, allow for more complex user interactions and dynamic experiences. Using these tools, developers can build out light web applications without ever configuring or managing a server. This elevates websites from static flyers to a modern, personalized customer experience without piling on excess developer work.
While every content management system will have its advantages, CMS Hub also includes a built-in relational database, multi-language support, and the ability to build dynamic content and login pages based on CRM data. All features designed to make life easier for developers.
Modern CMS-Based Websites Make Marketers Happy, Too
Marketing teams can immediately take advantage of CMS features, especially when using a CRM-powered solution. They can add pages, edit copy, and even alter styling using a drag-and-drop editor, without needing help from a busy developer. This empowers the marketing team and reduces friction when making updates. It also reduces the volume of support requests that developers have to manage.
Marketers can also benefit from built-in tools for search engine optimization (SEO), A/B testing, and specialized analytics. In addition to standard information like page views, a CRM-powered website offers contact attribution reporting. This end-to-end measurement reveals which initiatives generate actual leads via the website. These leads then flow seamlessly into the CRM for the sales team to close deals.
CRM-powered websites also support highly customized experiences for site users. The CRM behind the website already holds the customer data. This data automatically synchronizes because it lives within one system as a single source of truth for both marketing pages and sales workflows. This default integration saves development teams time that they would otherwise spend building data pipelines.
Next Steps
Every situation is unique, and in some cases, a static site generator is the right decision. But if you are building a site for an organization and solving for the needs of developers and marketers, a modern CMS may be the way to go.
Options like CMS Hub offer all the benefits of a content management system while coming close to matching static site generators’ marquee features: page load speed, simple deployment, and stout reliability. But don’t take my word for it. Create a free CMS Hub developer test account and take it for a test drive.
You might already know about the CSS gap property. It isn’t exactly new, but it did gain an important new ability last year: it now works in Flexbox in addition to CSS Grid. That, and the fact that I believe the property is more complicated than it appears, made me want to go back and explain exactly how it works.
Let’s take a proper look at gap and its associated properties, and understand how and where they work.
All the gap properties
To start us off, let’s review all of the CSS properties related to gap. There are six in total:
grid-row-gap
grid-column-gap
grid-gap
row-gap
column-gap
gap
From this list, we can already ignore the first three properties. The grid-* properties were added early when CSS Grid’s specifications were being drafted, and later deprecated when gap became more generalized. Browsers still support these deprecated grid-* properties (as of this writing) and merely treat them as if the grid- prefix is not present. Hence, grid-gap is the same as gap, grid-column-gap is the same as column-gap and grid-row-gap is the same as row-gap.
As for the other three properties, knowing that gap is a shorthand that lets you specify the other two properties, we really only need to know what row-gap and column-gap do.
Our understanding of these properties depends on the type of CSS layout we’re using. Let’s look at those options first.
Where can gaps be used?
If you’re like me, you’ve used gaps in grid layouts, but they can now be used in Flexbox, as well as multi-column layouts. Let’s go over each case.
Grid gaps
All browsers support gaps in grid layouts, and they’re quite simple to understand in this context.
row-gap introduces space between row tracks
column-gap introduces space between column tracks
Let’s create a grid with three columns and two rows:
The lines in the picture above are called grid lines, and they separate the tracks (rows and columns) of the grid. These lines don’t really exist in the grid — they’re invisible, have no thickness, and are typically what DevTools displays when we enable the grid inspector (in Safari, Firefox, Edge or Chrome).
If we do, however, start adding gaps to our grid, it will work as though these lines start acquiring thickness.
The lines between our tracks are now 20px thick and therefore push grid items further apart.
It’s worth noting that the tracks still have the same sizes (defined by the grid-template-* properties); therefore, the grid is wider and taller than it was without gaps.
In grid, row-gap always applies between row tracks. So, if we replace gap with row-gap in the above example, we get this:
And column-gap always applies between column tracks, so replacing gap with column-gap produces the following result:
Grid is simple because, by default, columns are vertical, and rows are horizontal, just like in a table. So it’s easy to remember where column-gap and row-gap apply.
Now, things do get a little more complicated when writing-mode is used. The default writing mode on the web is horizontal, from left to right, but there are vertical writing modes as well, and when that happens, columns become horizontal and rows are vertical. Always pay attention to writing-mode as it can make it less intuitive than it usually is.
This is a good transition into the next section as columns and rows get new meanings within Flexbox.
Flexbox gaps
Let’s talk about gaps in Flexbox layouts, where things get a little more complicated. We’ll use the following example:
.container { display: flex; }
By default, this gives us a row flex container, which means items within the container are stacked from left to right on the same horizontal line.
In this case, column-gap is applied between items and row-gap does nothing. That’s because there is only one line (or row). But now let’s add some gap between items:
.container { display: flex; column-gap: 10px; }
Now let’s switch the flex-direction of our container to column, which stacks items vertically, from top to bottom, with the following code:
The gap disappeared. Even if column-gap did add space between items when the container was in a row direction, it does not work anymore in the column direction.
We need to use row-gap to get it back. Alternatively, we could use the gap shorthand with one value, which would apply the same gap in both directions and, therefore, work in both cases.
So, to summarize, colum-gap always works vertically (assuming the default writing-mode), and row-gap always works horizontally. This does not depend on the direction of the flex container.
But now take a look at an example where line wrapping is involved:
Here, we’re allowing items to wrap on multiple lines with flex-wrap: wrap if there isn’t enough space to fit everything in a single line.
In this case, the column-gap is still applied vertically between items, and row-gap is applied horizontally between the two flex lines.
There’s one interesting difference between this and grid. The column gaps don’t necessarily align across flex lines. That’s because of justify-content: center, which centers items within their flex lines. This way, we can see that each flex line is a separate layout where gaps apply independently of other lines.
Multi-column gaps
Multi-column is a type of layout that makes it very easy to automatically flow content between several columns, like what you might expect in a traditional newspaper article. We set a number of columns and set the size for each column.
Gaps in multi-column layouts don’t quite work the same as grid or Flexbox. There are three notable differences:
row-gap has no effect,
column-gap has a non-0 default value,
and gaps can be styled.
Let’s break those down. First of all, row-gap has no effect. In multi-column layouts, there aren’t any rows to separate. That means only column-gap is relevant (as is the gap shorthand).
Secondly, unlike in grid and Flexbox, column-gap has a default value of 1em in multi-column layouts (as opposed to 0). So, even when no gap is specified at all, the columns of content are still visually separated. The default gap can, of course, be overridden but it’s a good default to have.
Here is the code that the example is based on:
.container { column-count: 3; padding: 1em; }
Finally, we can style the empty gap between columns in a multi-column layout. We use the column-rule property which works like border:
The column-rule property gives us some styling affordance in a multi-column layout.
Browser support
gap is really well-supported across the board. There’s more information over at caniuse, but to summarize:
Flexbox:gap is supported everywhere except for Internet Explorer (which is on its way out), Opera Mini and UC Browser for Android. caniuse has global support at 87.31%.
Multi-column: Same thing, too, but it’s unsupported in Safari and has 75.59% global support.
So, overall, the gap property is well supported and, in most cases, workarounds are unnecessary.
Styling the gap in flex and grid
Styling gap in Flexbox and CSS Grid would be really useful. The sad news is that it isn’t supported anywhere today. But the good news is that it could be in the near future. This has been discussed over at the CSS working group and is in the works in Firefox. Once we have a working implementation in Firefox along with the spec proposal, perhaps it will drive implementation in other browsers.
In the meantime, there are ways around this.
One is to give the grid container a background color, then a different color for the items, and finally a gap to make the container color show through.
While this works, it means we’re unable to use gaps to introduce space between items. The gap here acts as a border width instead. So, to visually separate the items out a bit more, we need to use padding or margin on the items, which isn’t as great… as we’ll see in the next section.
Can’t I just use margin or padding?
Yes, in most cases we can also use margin (and/or padding) to add visual space between elements of a layout. But gap comes with multiple advantages.
First, gaps are defined at the container level. This means we define them once for the entire layout and they are applied consistently within it. Using margins would require a declaration on each and every item. This can get complicated when items are different in nature, or come from different reusable components.
On top of this, gaps do the right thing by default with just one line of code. For example, if we’re trying to introduce some space in between flex items, not around them, margin would require special cases to remove extra margins before the firstitemand after the last one. With gaps, we don’t need to do this.
With a margin: 0 20px on each flex item, we’d end up with:
However with a gap: 40px on the container, we’d get this:
Similarly in grid layout, defining gap at the container level is much simpler and provides better results than having to define a margin on each item and accounting for the margin that applies on the edge of the grid.
With margin: 20px on each grid item:
And with gap: 40px on the grid container instead:
Empty space adds up
With everything said up to this point, margin and gap don’t have to be exclusive. In fact, there are many ways to spread items of a layout further apart, and they all can work together very well.
The gap property is just one part of the empty space created between boxes in a layout container. margin, padding, and alignment all may increase the empty space on top of what gap already defines.
Let’s consider an example where we build a simple flex layout with a given width, some gap, some distribution of content (using justify-content), and some margin and padding:
Let’s assume this code produces the following result:
Now, let’s see exactly how the empty space between items got created:
As we see, there are four different types of empty space between two consecutive flex items:
Between two consecutive items, the gap defines the minimum space between these items. There can be more, like in this case, but there can never be less space.
Margin pushes items even further apart, but unlike gap, it adds space both sides of all items.
Padding provides some space inside each item.
Finally, and only because there is enough remaining space, content distribution kicks in and distributes the items evenly within the flex line, according to the space-around value.
Debugging gaps
Let’s conclude with a topic that’s very near and dear to my heart: DevTools support for debugging gaps. There can always be cases where things go wrong, and knowing that DevTools has got our backs is very comforting, but we do need to know which tools can help us in this case.
For gap, I can think of two very specific features that might become useful.
Is my gap active?
Unless we misspelled gap or provided an invalid value, the property is always going to apply to the page. For example, this is correct:
.some-class { display: block; gap: 3em; }
It won’t do anything, but it is valid CSS and the browser doesn’t mind that gap doesn’t apply to block layouts. However, Firefox has a feature called Inactive CSS that does just this: it cares about valid CSS that’s applied to things that make sense. In this case, Firefox DevTools displays a warning in the Inspector.
Where is my gap?
Chrome and Microsoft Edge also have a very useful feature for debugging gaps. It was added through a collaboration between Microsoft and Google that was aimed at building layout debugging tools in Chromium (the open source project that powers both of the browsers, as well as others). In these browsers, you can hover over individual properties in the Styles panel, and see their effect on the page.
The cursor is hovering over the gap and the justify-content properties in the Styles panel, and the corresponding areas of the page light up to indicate where these properties have effect.The cursor is hovering over the margin and padding properties, which highlights the corresponding box model areas of the page.
And that’s it. I hope this article was useful in helping understand some of the details of how gaps work in CSS.
Here’s a common thought and question: how do browsers prioritize what they work on? We get little glimpses of it sometimes. We’re told to “star issues” in bug trackers to signal interest. We’re told to get involved in GitHub threads for spec issues. We’re told they do read the blog posts. And, sometimes, we get to see the results of surveys. Chrome ran a survey about scrolling on the web back in April and has published the results with an accompanying a blog post.
“Scrolling” is a big landscape:
From our research, these difficulties come from the multitude of use cases for scroll. When we talk about scrolling, that might include:
According to the results, dang near half of developers are dissatisfied with scrolling on the web, so this is a metric Google devs want to change and they will prioritize it.
To add to the list above, I think even smooth scrolling is a little frustrating in how you can’t control the speed or other behaviors of it. For example, you can’t say “smooth scroll an on-page jump-down link, but don’t smooth scroll a find-on-page jump.”
And that’s not to mention scroll snapping, which is another whole thing with the occasional bug. Speaking of which, Dave had an idea on the show the other day that was pretty interesting. Now that scroll snapping is largely supported, even on desktop, and feels pretty smooth for the most part, should we start using it more liberally, like on whole page sections? Maybe even like…
I’ve certainly seen scroll snapping in more places. Like this example from Scott Jehl where he was playing with scroll snapping on fixed table headers and columns. It’s a very nice touch:
It’s not every day that a new pattern emerges across the web, but I think cmd + k is here to stay. It’s a keyboard shortcut that usually pops open a search UI and it lets you toggle settings on or off, such as dark mode. And lots of apps support it now—Slack, Notion, Linear, and Sentry (my current gig) are the ones that I’ve noticed lately, but I’m sure tons of others have started picking up on this pattern.
kbar is a fully extensible command+k interface for your site
My only hope is that more websites and applications start to support it in the future—with kbar being a great tool to help spread the good word about this shortcut.
Have you ever been in a situation where you wish you could have some control over the values in an object or array? Maybe you wanted to prevent certain types of data or even validate the data before storing it in the object. Suppose you wanted to react to the incoming data in some way, or even the outgoing data? For example, maybe you wanted to update the DOM by displaying results or swap classes for styling changes as data changes. Ever wanted to work on a simple idea or section of page that needed some of the features of a framework, like Vue or React, but didn’t want to start up a new app?
I’ll say up front: when it comes to front-end technologies, I’m more of a UI developer; much like described non-JavaScript-focused side of The Great Divide. I’m happy just creating nice-looking projects that are consistent in browsers and all the quirks that go with that. So when it comes to more pure JavaScript features, I tend not to go too deep.
Yet I still like to do research and I’m always looking for something to add to that list of new things to learn. Turns out JavaScript proxies are an interesting subject because just going over the basics opens up many possible ideas of how to leverage this feature. Despite that, at first glance, the code can get heavy quick. Of course, that all depends on what you need.
The concept of the proxy object has been with us for quite some time now. I could find references to it in my research going back several years. Yet it was not high on my list because it has never had support in Internet Explorer. In comparison, it has had excellent support across all the other browsers for years. This is one reason why Vue 3 isn’t compatible with Internet Explorer 11, because of the use of the proxy within the newest Vue project.
[…] enables you to create a proxy for another object, which can intercept and redefine fundamental operations for that object.
The general idea is that you can create an object that has functionality that lets you take control of typical operations that happen while using an object. The two most common would be getting and setting values stored in the object.
const myObj = { mykey: 'value' } console.log(myObj.mykey); // "gets" value of the key, outputs 'value' myObj.mykey = 'updated'; // "sets" value of the key, makes it 'updated'
So, in our proxy object we would create “traps” to intercept these operations and perform whatever functionality we might wish to accomplish. There are up to thirteen of these traps available. I’m not necessarily going to cover all these traps as not all of them are necessary for my simple examples that follow. Again, this depends on what you’re needing for the particular context of what you’re trying to create. Trust me, you can go a long way with just the basics.
To expand on our example above to create a proxy, we would do something like this:
const myObj = { mykey: 'value' } const handler = { get: function (target, prop) { return target[prop]; }, set: function (target, prop, value) { target[prop] = value; return true; } } const proxy = new Proxy(myObj, handler); console.log(proxy.mykey); // "gets" value of the key, outputs 'value' proxy.mykey = 'updated'; // "sets" value of the key, makes it 'updated'
First we start with our standard object. Then we create a handler object that holds the handler functions, often called traps. These represent the operations that can be done on a traditional object which, in this case, are the get and set that just pass things along with no changes. After that, we create our proxy using the constructor with our target object and the handler object. At that point, we can reference the proxy object in getting and setting values which will be a proxy to the original target object, myObj.
Note return true at the end of the set trap. That’s intended to inform the proxy that setting the value should be considered successful. In some situations where you wish to prevent a value being set (think of a validation error), you would return false instead. This would also cause a console error with a TypeError being outputted.
Now one thing to keep in mind with this pattern is that the original target object is still available. That means you could bypass the proxy and alter values of the object without the proxy. In my reading about using the Proxy object, I found useful patterns that can help with that.
let myObj = { mykey: 'value' } const handler = { get: function (target, prop) { return target[prop]; }, set: function (target, prop, value) { target[prop] = value; return true; } } myObj = new Proxy(myObj, handler); console.log(myObj.mykey); // "gets" value of the key, outputs 'value' myObj.mykey = 'updated'; // "sets" value of the key, makes it 'updated'
In this pattern, we’re using the target object as the proxy object while referencing the target object within the proxy constructor. Yeah, that happened. This works, but I found it somewhat easy to get confused over what’s happening. So let’s create the target object inside the proxy constructor instead:
const handler = { get: function (target, prop) { return target[prop]; }, set: function (target, prop, value) { target[prop] = value; return true; } } const proxy = new Proxy({ mykey: 'value' }, handler); console.log(proxy.mykey); // "gets" value of the key, outputs 'value' proxy.mykey = 'updated'; // "sets" value of the key, makes it 'updated'
For that matter, we could create both the target and handler objects inside the constructor if we prefer:
const proxy = new Proxy({ mykey: 'value' }, { get: function (target, prop) { return target[prop]; }, set: function (target, prop, value) { target[prop] = value; return true; } }); console.log(proxy.mykey); // "gets" value of the key, outputs 'value' proxy.mykey = 'updated'; // "sets" value of the key, makes it 'updated'
In fact, this is the most common pattern I use in my examples below. Thankfully, there is flexibility in how to create a proxy object. Just use whatever patterns suits you.
The following are some examples covering usage of the JavaScript Proxy from basic data validation up to updating form data with a fetch. Keep in mind these examples really do cover the basics of JavaScript Proxy; it can go deeper quick if you wish. In some cases they are just about creating regular JavaScript code doing regular JavaScript things within the proxy object. Look at them as ways to extend some common JavaScript tasks with more control over data.
A simple example for a simple question
My first example covers what I’ve always felt was a rather simplistic and strange coding interview question: reverse a string. I’ve never been a fan and never ask it when conducting an interview. Being someone that likes to go against the grain in this kind of thing, I played with outside-the-box solutions. You know, just to throw it out there sometimes for fun and one of these solutions is a good bit of front end fun. It also makes for a simple example showing a proxy in use.
If you type into the input you will see whatever is typed is printed out below, but reversed. Obviously, any of the many ways to reverse a string could be used here. Yet, let’s go over my strange way to do the reversal.
const reverse = new Proxy( { value: '' }, { set: function (target, prop, value) { target[prop] = value; document.querySelectorAll('[data-reverse]').forEach(item => { let el = document.createElement('div'); el.innerHTML = '' + value; item.innerText = el.innerHTML; }); return true; } } ) document.querySelector('input').addEventListener('input', e => { reverse.value = e.target.value; });
First, we create our new proxy and the target object is a single key value that holds whatever is typed into the input. The get trap isn’t there since we would just need a simple pass-through as we don’t have any real functionality tied to it. There’s no need to do anything in that case. We’ll get to that later.
For the set trap we do have a small bit of functionality to perform. There is still a simple pass-through where the value is set to the value key in the target object like normal. Then there is a querySelectorAll that finds all elements with a data-reverse data attribute on the page. This allows us to target multiple elements on the page and update them all in one go. This gives us our framework-like binding action that everybody likes to see. This could also be updated to target inputs to allow for a proper two-way binding type of situation.
This is where my little fun oddball way of reversing a string kicks in. A div is created in memory and then the innerHTML of the element is updated with a string. The first part of the string uses a special Unicode decimal code that actually reverses everything after, making it right-to-left. The innerText of the actual element on the page is then given the innerHTML of the div in memory. This runs each time something is entered into the input; therefore, all elements with the data-reverse attribute is updated.
Lastly, we set up an event listener on the input that sets the value key in our target object by the input’s value that is the target of the event.
In the end, a very simple example of performing a side effect on the page’s DOM through setting a value to the object.
Live-formatting an input value
A common UI pattern is to format the value of an input into a more exact sequence than just a string of letters and numbers. An example of this is an telephone input. Sometimes it just looks and feels better if the phone number being typed actually looks like a phone number. The trick though is that, when we format the input’s value, we probably still want an unformatted version of the data.
This is an easy task for a JavaScript Proxy.
As you type numbers into the input, they’re formatted into a standard U.S. phone number (e.g. (123) 456-7890). Notice, too, that the phone number is displayed in plain text underneath the input just like the reverse string example above. The button outputs both the formatted and unformatted versions of the data to the console.
There’s more code in this example, so let’s break it down. The first part is the target object that we are initializing inside the proxy itself. It has three things happening.
The first key, _clean, is our variable that holds the unformatted version of our data. It starts with the underscore with a traditional variable naming pattern of considering it “private.” We would like to make this unavailable under normal circumstances. There will be more to this as we go.
The second key, number, simply holds the formatted phone number value.
The third "key" is a get function using the name clean. This returns the value of our private _clean variable. In this case, we’re simply returning the value, but this provides the opportunity to do other things with it if we wish. This is like a proxy getter for the get function of the proxy. It seems strange but it makes for an easy way to control our data. Depending on your specific needs, this might be a rather simplistic way to handle this situation. It works for our simple example here but there could be other steps to take.
Now for the get trap of the proxy.
get: function (target, prop) { if (!prop.startsWith('_')) { return target[prop]; } else { return 'entry not found!' } },
First, we check for the incoming prop, or object key, to determine if it does not start with an underscore. If it does not start with an underscore, we simply return it. If it does, then we return a string saying the entry was not found. This type of negative return could be handled different ways depending on what is needed. Return a string, return an error, or run code with different side effects. It all depends on the situation.
One thing to note in my example is that I’m not handling other proxy traps that may come into play with what would be considered a private variable in the proxy. For a more complete protection of this data, you would have to consider other traps, such as [defineProperty](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy/Proxy/defineProperty), deleteProperty, or ownKeys — typically anything about manipulating or referring to object keys. Whether you go this far could depend on who would be making use of the proxy. If it’s for you, then you know how you are using the proxy. But if it’s someone else, you may want to consider locking things down as much as possible.
Now for where most of the magic happens for this example — the set trap:
First, the same check against the private variable we have in the proxy. I don’t really test for other types of props, but you might consider doing that here. I’m assuming only that the number key in the proxy target object will be adjusted.
The incoming value, the input’s value, is stripped of everything but number characters and saved to the _clean key. This value is then used throughout to rebuild into the formatted value. Basically, every time you type, the entire string is being rebuilt into the expected format, live. The substring method keeps the number locked down to ten digits.
Then a sections object is created to hold the different sections of our phone number based on the breakdown of a U.S. phone number. As the _clean variable increases in length, we update number to a formatting pattern we wish to see at that point in time.
A querySelectorAll is looking for any element that has the data-phone_number data attribute and run them through a forEach loop. If the element is an input where the value is updated, the innerText of anything else is updated. This is how the text appears underneath the input. If we were to place another input element with that data attribute, we would see its value updated in real time. This is a way to create one-way or two-way binding, depending on the requirements.
In the end, true is returned to let the proxy know everything went well. If the incoming prop, or key, starts with an underscore, then false is returned instead.
Finally, the event listeners that makes this work:
The first set finds all the inputs with our specific data attribute and adds an event listener to them. For each input event, the proxy’s number key value is updated with the current input’s value. Since we’re formatting the value of the input that gets sent along each time, we strip out any characters that are not numbers.
The second set finds the button that outputs both sets of data, as requested, to the console. This shows how we could write code that requests the data that is needed at any time. Hopefully it is clear that phone.clean is referring to our get proxy function that’s in the target object that returns the _clean variable in the object. Notice that it isn’t invoked as a function, like phone.clean(), since it behaves as a get proxy in our proxy.
Storing numbers in an array
Instead of an object you could use an array as the target “object” in the proxy. Since it would be an array there are some things to consider. Features of an array such as push() would be treated certain ways in the setter trap of the proxy. Plus, creating a custom function inside the target object concept doesn’t really work in this case. Yet, there are some useful things to be done with having an array as the target.
Sure, storing numbers in an array isn’t a new thing. Obviously. Yet I’m going to attach a few rules to this number-storing array, such as no repeating values and allowing only numbers. I’ll also provide some outputting options, such sort, sum, average, and clearing the values. Then update a small user interface that controls it all.
Here’s the proxy object:
const numbers = new Proxy([], { get: function (target, prop) { message.classList.remove('error'); if (prop === 'sort') return [...target].sort((a, b) => a - b); if (prop === 'sum') return [...target].reduce((a, b) => a + b); if (prop === 'average') return [...target].reduce((a, b) => a + b) / target.length; if (prop === 'clear') { message.innerText = `$ {target.length} number$ {target.length === 1 ? '' : 's'} cleared!`; target.splice(0, target.length); collection.innerText = target; } return target[prop]; }, set: function (target, prop, value) { if (prop === 'length') return true; dataInput.value = ''; message.classList.remove('error'); if (!Number.isInteger(value)) { console.error('Data provided is not a number!'); message.innerText = 'Data provided is not a number!'; message.classList.add('error'); return false; } if (target.includes(value)) { console.error(`Number $ {value} has already been submitted!`); message.innerText = `Number $ {value} has already been submitted!`; message.classList.add('error'); return false; } target[prop] = value; collection.innerText = target; message.innerText = `Number $ {value} added!`; return true; } });
With this example, I’ll start with the setter trap.
First thing to do is to check against the length property being set to the array. It just returns true so that it would happen the normal way. It could always have code in place in case reacting to the length being set if we needed.
The next two lines of code refer to two HTML elements on the page stored with a querySelector. The dataInput is the input element and we wish to clear it on every entry. The message is the element that holds responses to changes to the array. Since it has the concept of an error state, we make sure it is not in that state on every entry.
The first if checks to see if the entry is in fact a number. If it is not, then it does several things. It emits a console error stating the problem. The message element gets the same statement. Then the message is placed into an error state via a CSS class. Finally, it returns false which also causes the proxy to emit its own error to the console.
The second if checks to see if the entry already exists within the array; remember we do not want repeats. If there is a repeat, then the same messaging happens as in the first if. The messaging is a bit different as it’s a template literal so we can see the repeated value.
The last section assumes everything has gone well and things can proceed. The value is set as usual and then we update the collection list. The collection is referring to another element on the page that shows us the current collection of numbers in the array. Again, the message is updated with the entry that was added. Finally, we return true to let the proxy know all is well.
Now, the get trap is a bit different than the previous examples.
get: function (target, prop) { message.classList.remove('error'); if (prop === 'sort') return [...target].sort((a, b) => a - b); if (prop === 'sum') return [...target].reduce((a, b) => a + b); if (prop === 'average') return [...target].reduce((a, b) => a + b) / target.length; if (prop === 'clear') { message.innerText = `$ {target.length} number$ {target.length === 1 ? '' : 's'} cleared!`; target.splice(0, target.length); collection.innerText = target; } return target[prop]; },
What’s going on here is taking advantage of a “prop” that’s not a normal array method; it gets passed along to the get trap as the prop. Take for instance the first “prop” is triggered by this event listener:
So when the sort button is clicked, the message element’s innerText is updated with whatever numbers.sort returns. It acts as a getter that the proxy intercepts and returns something other than typical array-related results.
After removing the potential error state of the message element, we then figure out if something other than a standard array get operation is expected to happen. Each one returns a manipulation of the original array data without altering the original array. This is done by using the spread operator on the target to create a new array and then standard array methods are used. Each name should suggest what it does: sort, sum, average, and clear. Well, OK, clear isn’t exactly a standard array method, but it sounds good. Since the entries can be in any order, we can have it give us the sorted list or do math functions on the entries. Clearing simply wipes out the array as you might expect.
Here are the other event listeners used for the buttons:
There are many ways we could extend and add features to an array. I’ve seen examples of an array that allows selecting an entry with a negative index that counts from the end. Finding an entry in an array of objects based on a property value within an object. Have a message returned on trying to get a nonexistent value within the array instead of undefined. There are lots of ideas that can be leveraged and explored with a proxy on an array.
Interactive address form
An address form is a fairly standard thing to have on a web page. Let’s add a bit of interactivity to it for fun (and non-standard) confirmation. It can also act as a data collection of the values of the form within a single object that can be requested on demand.
The target object is quite simple; the entries for each input in the form. The getData function will return the object but if a property has an empty string for a value it will change to “no entry!” This is optional but the function gives a cleaner object than what we would get by just getting the state of the proxy object.
The getter function simply passes things along as usual. You could probably do without that, but I like to include it for completeness.
The setter function sets the value to the prop. The if, however, checks to see if the prop being set happens to be the zip code. If it is, then we check to see if the length of the value is five. When the evaluation is true, we perform a fetch that hits an address finder API using the zip code. Any values that are returned are inserted into the object properties, the city input, and selects the state in the select element. This an example of a handy shortcut to let people skip having to type those values. The values can be changed manually, if needed.
For the next section, let’s look at an example of an input element:
The proxy has a querySelectorAll that looks for any elements that have a matching data attribute. This is the same as the reverse string example we saw earlier. If it finds a match, it updates either the input’s value or element’s innerText. This is how the rotated card is updated in real-time to show what the completed address will look like.
One thing to note is the data-model attribute on the inputs. The value of that data attribute actually informs the proxy what key to latch onto during its operations. The proxy finds the elements involved based on that key involves. The event listener does much the same by letting the proxy know which key is in play. Here’s what that looks like:
So, all the inputs within the main element are targeted and, when the input event is fired, the proxy is updated. The value of the data-model attribute is used to determine what key to target in the proxy. In effect, we have a model-like system in play. Think of ways such a thing could be leveraged even further.
As for the “get data” button? It’s a simple console log of the getData function…
This was a fun example to build and use to explore the concept. This is the kind of example that gets me thinking about what I could build with the JavaScript Proxy. Sometimes, you just want a small widget that has some data collection/protection and ability to manipulate the DOM just by interacting with data. Yes, you could go with Vue or React, but sometimes even they can be too much for such a simple thing.
That’s all, for now
“For now” meaning that could depend on each of you and whether you’ll dig a bit deeper into the JavaScript Proxy. Like I said at the beginning of this article, I only cover the basics of this feature. There is a great deal more it can offer and it can go bigger than the examples I’ve provided. In some cases it could provide the basis of a small helper for a niche solution. It’s obvious that the examples could easily be created with basic functions doing much the same functionality. Even most of my example code is regular JavaScript mixed with the proxy object.
The point though is to offer examples of using the proxy to show how one could react to interactions to data — even control how to react to those interactions to protect data, validate data, manipulate the DOM, and fetch new data — all based on someone trying to save or get the data. In the long run, this can be very powerful and allow for simple apps that may not warrant a larger library or framework.
So, if you’re a front-end developer that focuses more on the UI side of things, like myself, you can explore a bit of the basics to see if there are smaller projects that could benefit from JavaScript Proxy. If you’re more of a JavaScript developer, then you can start digging deeper into the proxy for larger projects. Maybe a new framework or library?
Locate a recipient from its Internet address,
. Since each hex digit can have 16 different values, the bank has enough space for 164 = 65,536 different subnets. Facebook, being a larger organization, was granted a prefix with room for over 4 billion subnets!
