Content Delivery Network (CDN)

Content delivery network (CDN)

Adding a cache tier fixed the repeated database reads. The web servers still do one slow thing on every page load: they ship the static files. A single page pulls a logo, a few images, a stylesheet or two, a bundle of JavaScript, maybe a video. None of that changes per user, and none of it has to come from your servers.

A CDN is a network of servers spread across many locations that serves those static files for you. You upload the files once; the CDN copies them out to its edge servers, and each user is served from the edge server nearest to them.

Why distance matters

A request to your origin server crosses the physical distance between the user and your data center, and back, for every file. If your origin is in San Francisco, a user in Los Angeles gets their files quickly, a user in Europe waits noticeably longer, and a user in Australia waits longer still. The bytes travel at a fixed speed and every extra mile is added latency.

An edge server near the user cuts that round trip down to a short local hop:

The direct fetch from the origin takes the full long-distance round trip. The same file from a nearby CDN node is an order of magnitude faster, and the CDN only pays the long trip to the origin once, the first time that file is asked for.

Static content only

This is for static content: images, video, CSS, JavaScript, fonts, anything that is the same bytes for every user. A CDN here is a store for files that do not change from one request to the next.

How a CDN serves a file

The file's URL points at a domain the CDN provider gives you, not at your own domain. Real examples of what those URLs look like on Amazon CloudFront and Akamai:

• https://mysite.cloudfront.net/logo.jpg
• https://mysite.akamai.com/image-manager/img/logo.jpg

The flow the first time a file is requested, and every time after:

1. User A requests image.png through its CDN URL.
2. The CDN edge server does not have image.png yet, so it requests the file from the origin. The origin can be a web server or object storage such as Amazon S3.
3. The origin returns image.png. The response carries an optional Time-to-Live (TTL) header saying how long the file may be cached.
4. The CDN stores the file and returns it to User A. It stays cached at that edge until the TTL runs out.
5. Later, User B requests the same file.
6. It comes straight back from the edge cache, no origin call, as long as the TTL has not expired.

So the first request for a file at a given edge is a cache miss and pays the origin round trip. Every request after that, until the TTL expires, is a hit served locally.

A page load, end to end

Put the pieces together with one real page load. You open www.mysite.com.

1. Your browser needs an IP for www.mysite.com. It asks DNS, which returns the address of the load balancer in front of the web tier. The browser connects there and asks for the page.
2. A web server builds the HTML. It reads what it needs from the cache and the database, renders the page, and sends it back. This response is different for every user, so the CDN plays no part in it.
3. The HTML does not contain the images and scripts, only links to them, and those links point at a different hostname:

<script src="https://static.mysite.com/app.4f9c2a.js"></script>
<img src="https://static.mysite.com/hero.jpg">

static.mysite.com is a CDN hostname.

4. Your browser asks DNS for static.mysite.com. That name is handed to the CDN, which picks an edge server near you and returns its address. The browser requests app.4f9c2a.js from that edge.
5. The edge checks its local copy. On a hit, the file comes back from a server a few milliseconds away. On a miss, the edge fetches it from the origin once, caches it, and returns it, and every request after that in your area gets the fast local copy.
6. The video on the page works the same way. The player pulls it in small chunks from a nearby edge, one chunk at a time as it plays.
7. Anything that changes while you use the page, a search, a comment, a fresh set of recommendations, is a separate request back to api.mysite.com, which resolves to the origin. The CDN does not answer these, because the response depends on who you are.

The hostname is the tell. Assets load from a CDN hostname and come from a server near you. The page itself and the API calls use the app hostname and come from the origin.

Considerations of using a CDN

• Cost. CDNs are run by third parties and bill you for data transferred in and out. A file that is rarely requested gets little benefit from being on the CDN and still costs you transfer fees, so it is worth pulling infrequently used assets back off the CDN.
• Setting an appropriate cache expiry. The TTL is a freshness tradeoff. Too long, and the CDN keeps serving an old copy after you have changed the file at the origin. Too short, and edges keep re-fetching from the origin before anyone reuses the cached copy, which wastes the CDN's main advantage. Time-sensitive content needs a deliberate TTL, not a default.
• CDN fallback. Decide what your site does when the CDN is down. During a CDN outage the client should notice the failure and fall back to requesting the files from the origin directly, so a CDN problem degrades performance instead of taking the site offline.
• Invalidating files. To remove a file from the CDN before its TTL expires, you have two options:
  • Call the invalidation API your CDN provider exposes, which tells the edges to drop that object.
  • Use object versioning: change the URL so it points at a new object. Add a version parameter to the query string, for example image.png?v=2. The old URL stays cached and untouched; the new URL is a fresh miss that fills from the origin.

Beyond static files: dynamic content

Everything above treats the CDN as a store for static files. It can do more, and this is extra depth on top of the design we build here.

CDNs can also cache dynamic responses, like an HTML page that is put together differently for each visitor. The way they do it is with a cache key. For a static file the key is just the file's URL, so every request for that URL gets the same cached copy. For a dynamic page the CDN builds the key from the parts of the request that actually change what comes back: usually the path, the query string, and a few named cookies or headers. Two requests that build the same key share one cached copy. A different key is a separate copy.

Dynamic pages go out of date quickly, so this is paired with short expiry times, or with an explicit purge the moment the underlying data changes. Many CDNs also let you run small pieces of code at the edge to build or personalize a page right there, instead of asking the origin every time.

For the rest of this series, the CDN holds static files and nothing else.

What is still fragile

Two failure modes are worth naming:

• A stale TTL means users see an old file until it expires, which is why invalidation and versioning exist.
• A CDN outage means every static request falls through to your origin at once, which is why the client needs an origin fallback path.

There is also a problem the CDN does not touch. The web servers still keep each user's session state in their own memory, so once a user's session lands on Server 1, every later request from that user has to go back to Server 1. That pins users to servers and makes the web tier harder to scale and to recover. Fixing it is the next post, on the stateless web tier.

The design so far

The cache absorbs repeated database reads, and the CDN now serves every static file from an edge near the user:

1. Static assets (JavaScript, CSS, images, video) are no longer served by the web servers. The browser fetches them from the CDN, from an edge server close to the user.
2. Dynamic requests still go through the load balancer to a web server.
3. The web server reads through the cache and only touches a slave database on a miss, so the database load is much lighter than before.
4. Writes still go to the master database.