The write path

Q: What databases do instead?

Databases do not fsync every page. They write sequentially to a log and fsync that, because sequential writes plus one flush are far cheaper than scattered writes plus many. The data pages are updated in memory and written back lazily. On crash, the log is replayed to reconstruct anything not yet written. This is write-ahead logging and it is the reason a database can be both fast and durable. Group commit takes it further: many concurrent transactions share one fsync. Under load the per-transaction cost of durability approaches zero, which is why database throughput often improves with concurrency up to a point.

Your code calls write(), gets no error, and moves on. Somewhere between there and a platter or a flash cell, there are five layers of buffering, and each one is a place where a power failure loses your data.

Most engineers have never traced this path, and most systems are less durable than their authors believe.

the layers#

Your application's buffer. Language-level buffered I/O. write() to a buffered stream may put bytes in userspace memory and return. A process crash loses them; the OS never saw them.

The page cache. An unbuffered write(2) reaches the kernel and returns once the data is in the page cache. This survives a process crash — the kernel still has it — but not a power loss. This is the layer most people mistake for durability.

fsync. Asks the kernel to push the file's pages to the device and wait. This is the call that means "durable," and it is the one people omit because it is slow.

The device's write cache. Consumer SSDs and disks acknowledge writes into a volatile cache. fsync normally issues a flush the device honours — but not always, and historically some devices lied about it. Enterprise drives have power-loss protection: a capacitor that lets the cache drain on power failure. This is a large part of what you pay for.

The filesystem's own metadata. Writing data durably is not enough if the directory entry pointing at it is not. This is why creating a file safely requires fsync on the directory as well, which almost nobody does.

the safe-write dance#

Writing a file so that a crash leaves either the old version or the new one, and never a truncated one:

1. write to a temporary file in the same directory
2. fsync(tmp_fd)               ← data is durable
3. rename(tmp, target)         ← atomic on POSIX
4. fsync(dir_fd)               ← the rename is durable
5. close everything

Steps 2 and 4 are the ones that get skipped. Without step 2 you can rename an empty file over a good one. Without step 4 the rename itself can be lost, leaving the old file — which is survivable — or, on some filesystems and configurations, neither.

This sequence is why "just write the config file" is harder than it looks, and why every mature piece of software that persists state has a function that does exactly this.

what databases do instead#

Databases do not fsync every page. They write sequentially to a log and fsync that, because sequential writes plus one flush are far cheaper than scattered writes plus many.

The data pages are updated in memory and written back lazily. On crash, the log is replayed to reconstruct anything not yet written. This is write-ahead logging and it is the reason a database can be both fast and durable.

Group commit takes it further: many concurrent transactions share one fsync. Under load the per-transaction cost of durability approaches zero, which is why database throughput often improves with concurrency up to a point.

the settings people change without understanding#

Postgres:

fsync = off — the database stops flushing. Fast, and a crash means restore-from-backup, not recover. Only ever appropriate for tests.
synchronous_commit = off — commits return before the WAL is durable. Bounded data loss on crash (a fraction of a second of transactions), no corruption. This is a genuine and reasonable trade for some workloads, and it is very different from fsync = off even though they sound similar.

MySQL:

innodb_flush_log_at_trx_commit = 2 — flush every second rather than every commit. Same shape of trade.

The distinction that matters: losing recent transactions is recoverable and sometimes acceptable. Corrupting the database is neither. The settings that risk the first are a tuning decision; the ones that risk the second are not.

replication is not durability#

A common and expensive misunderstanding.

Asynchronous replication means the primary acknowledges before the replica has the data. Primary dies, unreplicated transactions are gone — even though you have a replica, and even though it was "up to date" a second ago.

Synchronous replication waits for the replica, which is a real durability guarantee and a real latency cost.

Both are legitimate. Knowing which one you have is not optional, and a surprising number of teams find out during a failover.

the test nobody runs#

Write data. Confirm the application reported success. Cut the power — not a graceful shutdown, not kill -9, the actual power.

Bring it back. Is the data there?

For anything where losing writes is unacceptable, that test is the only thing that distinguishes a durability claim from a durability assumption. Cloud instances make it awkward to run, which is precisely why almost nobody does.