performance vs correctness

stronger guarantees cost latency, throughput, or both. faster systems get faster by accepting weaker ones. almost every architectural argument is a disagreement about where on that line a particular operation belongs.

why the tradeoff is structural

correctness requires coordination. if two nodes must agree on a value before returning it, they exchange messages and wait. that wait is latency, and while they are waiting they are not processing anything else, which is throughput.

performance comes from skipping coordination. serve a read from a local replica without checking whether it is current. acknowledge a write before every copy has it. answer with what you know instead of what you can prove.

this is not an implementation problem waiting for a better engineer. coordination costs round trips and round trips cost time. you can move the cost around, amortize it, or hide it behind a cache, but the physics stays.

what it costs, measured

the abstraction gets much clearer with a number on it. postgres exposes durability as a single setting, so the tradeoff can be measured directly rather than argued about:

$ psql -U postgres -c "SELECT name, setting, enumvals FROM pg_settings
                       WHERE name = 'synchronous_commit';"
        name        | setting |                 enumvals
--------------------+---------+-------------------------------------------
 synchronous_commit | on      | {local,remote_write,remote_apply,on,off}

that enum is a durability ladder, weakest to strongest: off returns before the WAL is on disk, local waits for a local flush, remote_write waits for a standby to receive it, on waits for a standby to flush it, remote_apply waits until it is visible to queries on the standby. the postgres docs on asynchronous commit spell out exactly what each one gives up.

the same workload against PostgreSQL 17.10, 8 clients, 20 seconds, only the setting changed:

######## synchronous_commit = on
number of transactions actually processed: 283556
latency average = 0.564 ms
tps = 14181.958150 (without initial connection time)

######## synchronous_commit = off
number of transactions actually processed: 1570449
latency average = 0.102 ms
tps = 78551.836242 (without initial connection time)

5.5x the throughput and a fifth of the latency, from one setting. nothing was optimized. the fast configuration is simply not waiting for the disk.

what you bought with that speed is a window: with off, postgres reports success for transactions whose WAL records are still in memory, and a crash in that window loses committed transactions. the database stays internally consistent, it just forgets things it told you it had saved.

so the question is never "is 5.5x worth it". it is "is losing the last fraction of a second of acknowledged writes acceptable for this data". for a page-view counter, obviously. for a payment, obviously not. the same server can run both if you set synchronous_commit per transaction, which is the point. durability is a per-write decision that most systems make once, globally, by accident.

correctness has rungs

consistency is not a boolean, and the models between "linearizable" and "eventual" are where most real systems live.

each rung up buys a stronger guarantee and charges round trips for it on every operation, including when nothing is wrong. that steady-state cost is the ELC half of PACELC and it dominates the partition case simply because partitions are rare and tuesdays are not.

the rung that is underrated is read-your-writes. a large share of user-visible consistency bugs are one person not seeing their own edit, and that is fixable with sticky routing rather than global consensus. you get most of the perceived correctness for a fraction of the coordination.

performance has dimensions too

optimizing one dimension routinely damages another.

latency is time for one request. caching helps it, coordination hurts it. throughput is requests per unit time; horizontal scaling helps, serialization hurts. these two are not the same axis. batching improves throughput by making individual requests wait.

tail latency is where the real problems hide. dean and barroso's the tail at scale makes the argument precisely: if a request fans out to 100 servers and each has a 1% chance of being slow, roughly 63% of requests hit at least one slow server. your p99 becomes the common case at the top level. this is why a change that improves mean latency can make the system feel worse, and why averages are close to useless as a service health signal.

be specific about which number you are moving. "faster" without a dimension usually means "better on the metric I happened to graph".

where it shows up

database reads. the primary has the latest data; a replica might be 50ms behind. a shopping cart tolerates that; the balance check immediately before a debit does not.

caching. a cache trades correctness for latency by definition, and the only question is the size of the staleness window. marc brooker's caches are hard is worth reading on why they also make failure modes worse. a cache that quietly absorbs load hides how much capacity you actually have until the moment it stops.

async processing. returning immediately and doing the work later is fast because the caller stops waiting. it also means you told the client "success" for work that has not happened, which buys you idempotency requirements, dead-letter queues, and reconciliation. event-driven architecture covers what that costs in full.

asking a better question

the common failure is picking a global default and applying it uniformly, usually the strongest one, because it feels responsible, or the weakest, because a benchmark looked good.

for any given operation, three questions get you most of the way:

what actually happens if this data is stale? who notices, and how fast? a like count that is two seconds old is invisible. an idempotency check reading from an eventually consistent store is a double charge.

what happens if an acknowledged write is lost? sometimes a re-sent analytics event. sometimes a customer's money.

who bears the cost of being wrong. you, or the user? that one tends to settle the argument.

the cost of being wrong varies enormously between components of the same system, which means uniform guarantees are almost always wrong in one direction or the other. design each part for the consistency it needs, not the strongest one you can afford to build.