Instagram should not be this fast

You decide to build Instagram. The first feature is the most boring one. A user uploads a photo with a caption. Later, somebody else opens the app, and that photo needs to be on their screen inside two hundred milliseconds. That is your budget. Less than the blink of an eye, from the moment they tap to the moment the image is rendering. Easy when one person is using your app. The trouble is that Instagram has two billion of them, and at least one of them is MrBeast.

I have built enough small things on top of Postgres to know the math of one user and the math of a million are not the same problem. A query that runs in a millisecond against a small table looks the same as a query that runs in a millisecond against a huge one, right up until the moment three million people ask for the same row inside the same minute. Then it stops looking the same. The naive version that runs your school project would not just be slow at Instagram scale. It would refuse to come up.

Every fast thing on your phone is a layered cheat, with one more layer added every time the audience got bigger and the old one stopped working.

The first answer is Postgres

The first solution anyone reaches for is a database. Postgres is the default. You make a posts table with columns id, author, caption, image_url, created_at. To show a post, you run a query.

SELECT id, author, caption, image_url, created_at
FROM posts
WHERE id = 'post-482';

One query, one row, one response. On my laptop this takes about a millisecond against a small Postgres database. If three people request that post, it takes three milliseconds total. If a hundred people request it at once, Postgres scoops them up in parallel and nobody notices.

This works because the database is doing almost no work. Look up an index, jump to a page, read one row, send it back. A modern Postgres on a normal machine handles thousands of these per second without sweating. For your school project this is more than enough. For a startup with a thousand users this is more than enough. You can run a real business on this, and many do.

The trouble starts when the number of people asking for the same row stops being three and starts being three million.

A million people on the same row breaks Postgres

When a popular account posts, the people pulling that one row are not spread out. They arrive together, in the same minute, asking for the same id. Three failure modes show up, in this order.

The first is the connection limit. Postgres only lets a fixed number of clients talk to it at once. The default is a hundred. You can tune it higher, but not infinitely, because each connection costs memory. Three million concurrent requests cannot all hold a connection. Most of them queue, and the queue grows faster than the database can drain it.

The second is the hot row problem. Even if the connections were free, every request is asking the database to read the same row. Postgres takes locks. It reads the same page off disk over and over. A row that everyone wants is, in database language, a hot key. Hot keys do not parallelise well. The database serialises around them whether you want it to or not.

The third is read amplification, the boring industrial name for doing the same lookup five million times when you only needed to do it once. The post has not changed. Nobody edited it. You are paying full database price for an answer that has not moved since the first request landed.

Once you see those three problems sitting in the same room, the shape of the next solution stops being a mystery. You need a thing that sits in front of the database and remembers the answer for everyone after the first person asked.

The next answer is Redis

Redis is an in-memory key-value store. That sentence is doing a lot of work, so let me unpack it.

Key-value store means you do not write queries against it. You hand it a string, like post:482, and it hands you back whatever value you stored under that string. No JOIN, no WHERE, no index. Just lookup by exact name. You can think of it as a giant Map or dict that lives outside your app.

In-memory means it does not write to disk on every operation. The whole working set lives in RAM. RAM is roughly a hundred thousand times faster than spinning disk and several hundred times faster than even a good SSD. A Redis lookup takes about a hundred microseconds, which is one ten-thousandth of a second. You can do this ten thousand times in the time it takes to blink.

So here is what happens for the popular post. When you tap, the app does not go to the database first. It asks Redis. Do you have post:482? If Redis says yes, the request is over in a millisecond and the database never gets touched. If Redis says no, then the app goes to the database, gets the row, hands it to Redis on the way back, and the next person to ask gets the cached copy.

async function getPost(id: string) {
  const cached = await redis.get(`post:${id}`);
  if (cached) return JSON.parse(cached);

  const row = await db.query('SELECT * FROM posts WHERE id = $1', [id]);
  await redis.set(`post:${id}`, JSON.stringify(row), { EX: 3600 });
  return row;
}

This pattern is called read-through caching. The first request pays full database price. Every request after pays cache price, which is so much cheaper it functionally rounds to zero. The hot post sits in Redis for the rest of the day, fielding millions of requests, while the database does the work the cache cannot help with. The personalised stuff. The writes. The bits that change for every viewer.

A small honesty note. Instagram’s own primary cache is a cousin of Redis called memcached, and Meta has spent more than a decade tuning it. Twitter and Reddit reach for Redis directly. The category is in-memory key-value cache, the two famous names are Redis and memcached, and the principle is identical whichever one is sitting in front of the database.

Then MrBeast posts and Redis is also not enough

A million people pulling the same row is the problem Redis fixes. The problem Redis cannot fix on its own is the same row being pulled at a scale a single cache box cannot serve.

MrBeast has hundreds of millions of followers across his accounts. When he posts, the volume of requests aimed at that one row is a different shape of problem. A single Redis node serves something like a hundred thousand to a million operations per second, depending on how it is tuned and what is on the line. A MrBeast post can pull ten million viewers inside the first few minutes, and the requests do not arrive evenly. They cluster in the first sixty seconds. The math does not fit on a single Redis box.

There is also the geography problem. If your Redis sits in Virginia and a user in Singapore taps the post, the round trip between them is roughly two hundred milliseconds before Redis even thinks about the answer. The latency budget for the whole feature was two hundred milliseconds. You are out of budget before you have done any work.

The answer for MrBeast is the edge

The trick at celebrity scale is that the post does not live in one place. It lives everywhere.

The image, the caption, the static parts of the response get pushed out to a content delivery network. A CDN is a fleet of servers spread across the world, each one holding a copy of the things people are likely to ask for. Cloudflare, Akamai, Fastly. When a user in Singapore taps a MrBeast post, the request never reaches Virginia. It hits a CDN node in Singapore, gets the post in under fifty milliseconds, and the central Redis and Postgres sleep through the whole thing.

The bits that cannot be cached at the edge are the personalised parts. Whether you liked it. The live like count, which changes every second. The comment thread. Those still go through Redis, then maybe through Postgres, but they are tiny pieces of JSON, kilobytes at most, and they fetch in parallel with the image that is already loading from the local CDN node. By the time the personalised payload arrives, the photo has been on your screen for a hundred milliseconds and you are halfway through the caption.

The CDN is not just a static-file server. It is a cache of the cache, spread across every continent, holding the answer ready before anyone asks.

What to take from this

The next time you load something instantly on a service you know has hundreds of millions of users, do not assume there is a magic database behind it. There almost certainly isn’t. There is a database. In front of it is a cache. In front of the cache is a globe of edge servers that handles the cases neither of the first two were built for.

Postgres stores the truth. Redis serves the speed. The CDN serves the celebrity. Each one was added the day the one before it stopped working. If you only learn the first layer in school, you will build a working version of Instagram for two hundred users and wonder what happened the day a real one shows up.

If you want to go deeper on the middle layer, the Redis documentation page on caching patterns is the cleanest primary source on the shape of the thing. Fair warning, it stops feeling overly simplified about thirty seconds in.