The Systems Book Everyone Recommends but Few Finish

I have a book on my shelf that I have started three times.

Designing Data-Intensive Applications.¹ People call it DDIA. If you have ever asked how to get better at backend systems, someone has told you to read it.

They were right. It is a very good book.

It is also around 600 dense pages, and most people who buy it never finish it.

I am one of them. Twice.

So this time I am doing something different. I am reading it in public, one part at a time, and writing down what actually sticks.

This post is the start of that series.

Why the book has a reputation

Most books about backend systems age badly. They pick a database, a framework, a cloud, and teach you that. Two years later the tool changed and the book is a museum piece.

DDIA does not do that.

It teaches the ideas underneath the tools. How data is stored. How it gets copied. How systems agree on what is true when the network is lying to them. The specific databases change. These problems do not.

That is why people keep recommending it. And it is why the book is hard. It is not a tutorial. It is closer to a survey of the whole field, written by someone who read the papers so you would not have to.²

You do not read it in a weekend.

You read it the way you would read a textbook. Slowly, and more than once.

Who it is really for

The title scares people. “Data-intensive” sounds like it means petabytes and a team of specialists.

It does not.

If your app has a database, a cache,³ a queue,⁴ or two services that talk to each other, you are building a data-intensive system. You already have most of the problems in this book. You just have smaller versions of them.

So here is who I think should read it:

• You write backend code and you want to understand what your database is actually doing.
• You keep hitting words like “replication lag” or “isolation level” and nodding without really knowing them.
• You want to make design decisions on purpose instead of by copying the last project.

And here is who can wait. If you are still learning to program, this is not your next book. Build some things first. Feel the pain. The book means more once you have been burned by the problems it describes.

Does the hype hold up? Mostly, yes. The one honest caveat: it is a 2017 book, so a few tool names are dated. The ideas are not. That trade is fine.

Chapter 1, in one breath

The first chapter is short, and it sets up everything else with three words.

Every system in the book is judged on three things.

Reliability. The system keeps working correctly even when things go wrong. And things go wrong. Disks die, networks drop, and people fat-finger a deploy.⁵ A reliable system expects faults and keeps its promises anyway.⁶

Scalability. The system has a reasonable answer for growth. What happens when load doubles? Which knob do you turn when it does? The chapter is careful here: describe your load with numbers, and measure performance with percentiles, not averages. Averages hide the users having the worst day.

Maintainability. A tired human can still understand and change the system six months from now. This is the one people skip, and it is the one that quietly decides whether a project survives.

That is the whole lens. Reliable, scalable, maintainable. Every later chapter is just a hard problem viewed through it.

How to read it without burning out

This is the part I got wrong the first two times.

I tried to read it front to back, absorb every detail, and follow every footnote into a research paper. I burned out around Chapter 5 both times.

So the plan this time is different.

Read for the shape, not for every detail. You do not need to memorize how one specific consensus algorithm works. You need to know that consensus is expensive and when you actually need it. The details are there when you want them later.

Read one part, then write. The book has three natural parts, and this series follows them. Writing forces me to find the idea I would repeat to a coworker. If I cannot explain it simply, I did not really get it.

Skip the footnotes on the first pass. They are gold, but they are also the trap. Follow them and you never finish the chapter.

Let it be slow. One post a week or two. Consistency beats speed. That is true for reading a hard book, and it turns out to be true for the systems in it.

One idea I’ll actually use

Every post in this series ends the same way: the one thing I am taking with me.

From Chapter 1, it is percentiles.

I used to describe how fast a system was with an average. The average looks great and lies to you. It smooths over the slow requests, and the slow requests are the ones a real person is sitting and waiting through.

From now on I am asking for the p95 and the p99.⁷ Not “how fast is it usually,” but “how bad is it for the unlucky ones.”

That is a small change in a metrics dashboard.

It is a large change in who you decide to care about.

Next in the series: how to choose a data model, and the project where I picked wrong.