Ask the right question before picking an answer

CP (consistency over availability)

When to use: Financial ledgers, inventory counts, anything where serving stale or conflicting data is worse than an error page.

Trade-offs: Nodes on the minority side of a partition refuse requests until they can confirm they have the latest state — real, user-visible downtime during a partition.

AP (availability over consistency)

When to use: Social feeds, shopping carts, presence indicators — anywhere a slightly stale read is acceptable and staying up matters more.

Trade-offs: Every node keeps serving during a partition, but different nodes can disagree until they reconcile — requires a conflict-resolution strategy (last-write-wins, CRDTs, application-level merge).

PACELC’s "else": latency vs consistency with no partition at all

When to use: Any system, at all times — this is the trade you’re making on the happy path, not just during failures.

Trade-offs: Synchronous replication for strong consistency adds round-trip latency to every write; async replication is fast but leaves a window where a reader can see stale data.

Monolith

When to use: Early-stage products, small teams, or domains still being discovered — one deployable, one datastore, simplest operational surface.

Trade-offs: Fast to build and reason about, but scaling and deploying one piece means scaling/deploying the whole thing.

Modular monolith

When to use: Team wants clear internal domain boundaries (for a future split) without paying distributed-systems tax today.

Trade-offs: Keeps deployment simple while enforcing module boundaries in code — the natural middle ground, and often underrated.

Microservices

When to use: Multiple teams need independent deploy cadences, or components have genuinely different scaling/reliability needs.

Trade-offs: Independent scaling and deployment, at the cost of network calls between services, distributed debugging, and real operational overhead.

Buy / adopt an existing solution

When to use: The capability is not a differentiator for this product and mature solutions already exist — auth, payments, rate limiting.

Trade-offs: Fast to ship and offloads ongoing maintenance to the vendor, but adds recurring cost and real vendor lock-in.

Build in-house

When to use: The capability IS the differentiator, or a hard constraint (e.g. a $0 budget) rules out every paid option.

Trade-offs: Full control and no recurring cost, but the team owns every bug, edge case, and security patch for as long as the system lives.

Hybrid — thin wrapper around an open-source or managed core

When to use: The common middle case: most of the problem is commodity, but a specific piece needs product-specific behavior.

Trade-offs: Balances speed and control, but needs an explicit customization boundary decided up front, or the "thin" wrapper quietly grows into a second implementation.

Compute-bound workloads

When to use: Heavy computation on data already in memory — encoding, dense ML inference, cryptographic hashing.

Trade-offs: More/faster cores and parallelization help directly, but Amdahl’s law caps the return on parallelizing a workload that has any sequential portion at all.

Memory/data-movement-bound workloads

When to use: The working set doesn’t fit in fast memory — large joins, wide scans, anything that spends more time shuffling data than computing on it.

Trade-offs: Better caching, columnar formats, and reducing data movement help; throwing more CPU cores at it does nothing if the cores are idle waiting on memory bandwidth.

Network-bound workloads

When to use: Latency dominated by round trips between services or regions, not by local computation — chatty microservices, cross-region calls.

Trade-offs: Batching requests and colocating services that talk to each other constantly cuts round trips; adding raw compute to either side does nothing about the wire time in between.

Disk/storage I/O-bound workloads

When to use: Random reads/writes against a dataset too large for memory — large transactional databases, log-heavy systems.

Trade-offs: Faster storage (SSD/NVMe) and higher IOPS provisioning help directly; more CPU or RAM elsewhere in the system won’t move data off disk any faster.

Enforce once, at the edge (API gateway / BFF)

When to use: Simple systems with a small number of trusted internal services and a single, well-defined entry point.

Trade-offs: Simplest to build and reason about, but every downstream service implicitly trusts the edge — a single compromised or misconfigured edge check is a total bypass for everything behind it.

Enforce at every layer independently (defense in depth)

When to use: Systems handling sensitive data, or with enough internal services that any one of them being wrong is a real risk.

Trade-offs: DB row-level security, service-level checks, and gateway-level checks all agreeing is much harder to bypass with a single mistake, but real duplication of authorization logic across layers, with a real risk the layers drift out of agreement over time.

Enforce at the service boundary that owns the data

When to use: Domain-driven systems where each service already owns a clear slice of data and wants to be the sole authority over who can touch it.

Trade-offs: Each service trusts only identity (who the caller is) from upstream, not authorization (what they’re allowed to do) — keeps authorization logic colocated with the data it protects, but means every service has to implement its own access checks rather than relying on a shared layer.

Load balancer (L4/L7)

When to use: Distributing requests across many identical instances of the same service — the default first line of defense against any single instance being overwhelmed.

Trade-offs: Simple and fast, especially at L4, but has no concept of "service A vs service B" — it balances, it doesn't route by business logic.

API gateway

When to use: A single entry point in front of many downstream services — centralizing auth, rate limiting, request/response transformation, and path-based routing.

Trade-offs: Powerful and centralizes cross-cutting concerns, but adds a network hop and becomes a shared dependency that must itself be scaled and kept highly available.

Reverse proxy

When to use: A general-purpose intermediary in front of one or more servers — TLS termination, response caching, compression, and hiding backend topology from the outside.

Trade-offs: Flexible and often free (nginx, Envoy), but on its own doesn't understand business-level routing the way a purpose-built gateway does.

DNS resolution

When to use: Every request starts here — domain-to-IP resolution, and (with geo/latency-based routing or Anycast) the layer where traffic gets steered to the nearest or healthiest region before a single packet reaches your infrastructure.

Trade-offs: TTL caching makes DNS-level failover slow to propagate — minutes, not seconds — so it's a coarse, not real-time, lever.

CDN / edge caching

When to use: Static or cacheable content served from a location physically close to the user — the single biggest latency win for content that doesn't change per-request.

Trade-offs: Excellent for cache hit rates, but cache invalidation and the static/dynamic content split need a real strategy or stale content ships silently — see the Reference Grid's CDN / edge entry.

Authentication (AuthN)

When to use: Establishing identity — session cookies for traditional web apps, JWTs for stateless/distributed APIs, OAuth for delegated third-party access.

Trade-offs: JWTs scale statelessly across instances but can't be revoked before expiry without an extra deny-list; sessions are trivially revocable but need shared or sticky state.

Authorization (AuthZ)

When to use: Once identity is established, deciding what that identity is actually allowed to do.

Trade-offs: RBAC/ABAC/ReBAC trade simplicity for flexibility in different ways — see the Reference Grid's AuthN vs AuthZ entry for the full comparison.

Security as the umbrella

When to use: The practices that protect the whole surface regardless of who's asking — TLS in transit, encryption at rest, input validation against the OWASP Top 10.

Trade-offs: None of these are optional trade-offs so much as a floor — see the Reference Grid's Transport encryption, Network security, and Input validation entries.

Eventual consistency

When to use: Analytics counters, follower counts, cache-backed reads — anywhere a few seconds of staleness is invisible to the user.

Trade-offs: Cheapest and most available option, but offers no ordering or freshness guarantee at all.

Monotonic reads

When to use: A user paging through results — guarantees they never see an older value after a newer one.

Trade-offs: Needs session/sticky routing to the same replica or a version token passed between requests.

Read-your-writes

When to use: A user edits their profile and immediately reloads the page — they must see their own edit.

Trade-offs: Typically requires routing that user’s reads to the primary (or a replica confirmed caught-up) right after a write.

Causal consistency

When to use: A comment reply must never render before the comment it replies to.

Trade-offs: Requires tracking causal metadata (vector clocks or similar) — more bookkeeping than read-your-writes.

Strong / linearizable consistency

When to use: Bank balance checks, distributed locks, leader election — anywhere every reader must agree on the single latest value.

Trade-offs: Highest coordination cost and latency of the spectrum; usually means routing everything through a single source of truth.

SQL / relational (ACID)

When to use: Orders, payments, anything with real relationships and a need for multi-table transactions to stay atomic.

Trade-offs: Strong guarantees and mature tooling, but schema changes are heavier and horizontal write-scaling is harder.

NoSQL / BASE (Basically Available, Soft state, Eventually consistent)

When to use: High-volume, loosely-structured, or rapidly-evolving data — session state, event logs, product catalogs with variable attributes.

Trade-offs: Scales out easily and schema is flexible, but you give up cross-record transactional guarantees by default.

Relational (Postgres/MySQL)

When to use: Structured data with relationships and transactional needs — the default unless something rules it out.

Trade-offs: Vertical scaling is the easy path; horizontal write-scaling needs sharding, which adds real complexity.

Key-value (Redis, DynamoDB)

When to use: Simple lookups by a known key at very high throughput — sessions, caches, feature flags.

Trade-offs: Extremely fast and simple, but no relational queries or joins.

Document (MongoDB)

When to use: Semi-structured, nested data that maps naturally to JSON and doesn’t need complex joins.

Trade-offs: Flexible schema speeds up iteration, but data duplication across documents can make updates harder to keep consistent.

Column-family (Cassandra, Bigtable)

When to use: Massive write throughput with known query patterns, e.g. time-series ingestion at scale.

Trade-offs: Excellent write scalability, but query flexibility is limited to patterns designed for upfront.

Search index (Elasticsearch)

When to use: Full-text search, fuzzy matching, faceted filtering.

Trade-offs: Best-in-class search UX, but it’s an eventually-consistent secondary index — not a system of record.

Graph (Neo4j)

When to use: Deeply relational, traversal-heavy queries — social graphs, recommendation paths.

Trade-offs: Graph traversals that would be expensive multi-join SQL queries become cheap, but it’s a less common tool with a smaller hiring pool.

Structured — relational or strongly-typed document store

When to use: Rows/columns or JSON with a stable, known shape — user records, orders, config.

Trade-offs: Enables indexing, joins, and validation at write time, at the cost of schema rigidity.

Unstructured — object storage / data lake

When to use: Logs, images, video, free-text documents — content without a fixed schema.

Trade-offs: Cheap, infinitely flexible storage, but querying internal structure requires a separate indexing/processing layer on top.

Semi-structured — search index or wide-column store

When to use: Text or logs that still need to be queried/filtered, not just archived.

Trade-offs: Bridges the gap, but is a secondary/derived store, not typically the system of record.

OLTP — row-oriented transactional database

When to use: The live product path: checkout, profile updates, inventory decrements.

Trade-offs: Optimized for fast single-row reads/writes, but large aggregate queries across millions of rows are slow and can degrade the transactional workload.

OLAP — column-oriented warehouse

When to use: Dashboards, BI reporting, large historical aggregations.

Trade-offs: Excellent at scanning/aggregating huge datasets, but not designed for high-frequency single-row transactional writes.

Index optimization

When to use: First lever, almost always — a missing or wrong index is the most common source of slow queries.

Trade-offs: Cheap and reversible, but only fixes query-shape problems, not raw volume.

Query optimization

When to use: After indexing — rewriting N+1 queries, reducing over-fetching.

Trade-offs: Still cheap, still reversible, still limited to inefficiency rather than true capacity limits.

Connection pooling

When to use: When connection overhead itself is the bottleneck under concurrent load.

Trade-offs: Low effort, meaningful win under high concurrency, doesn’t help raw data volume.

Caching

When to use: Read-heavy workloads with repeated queries for the same data.

Trade-offs: Big win for read latency and DB load, but introduces cache-invalidation complexity.

Read replicas

When to use: Read traffic outgrowing a single primary’s capacity.

Trade-offs: Scales reads horizontally, but replicas lag behind the primary — reintroduces the consistency-spectrum question.

Vertical scaling

When to use: A quick, temporary lever to buy time before a structural fix.

Trade-offs: Fast and simple, but has a hard ceiling and doesn’t address the underlying access pattern.

Sharding

When to use: Write volume has outgrown what a single primary can handle, after the cheaper levers are exhausted.

Trade-offs: Real horizontal write scaling, at the cost of cross-shard queries and transactions becoming much harder.

DB type change

When to use: The access pattern has fundamentally outgrown the current database category, not just its current scale.

Trade-offs: The most expensive lever — a migration — but sometimes the honest answer once every cheaper lever is spent.

FIFO

When to use: The simplest possible policy — a plain queue, evicting the oldest entry regardless of how hot it is.

Trade-offs: Trivial to implement, but ignores access pattern entirely — a frequently-hit entry gets evicted just as readily as a cold one.

LRU (Least Recently Used)

When to use: The default in practice — O(1) eviction via a hash map plus a doubly-linked list.

Trade-offs: Cheap and usually a good approximation of "hot," but a burst of one-off scans can evict an otherwise-hot working set — cache pollution.

LFU (Least Frequently Used)

When to use: Skewed/long-tail access patterns where LRU's cache pollution problem is a real, recurring issue.

Trade-offs: Resists that pollution problem by tracking access frequency, but needs a decay strategy — otherwise a once-popular-now-cold key pins itself in the cache forever.

TTL (Time To Live)

When to use: Data has a natural expiry — session tokens, rate-limit counters, anything that's wrong (not just stale) past a certain age.

Trade-offs: Correctness-driven rather than access-pattern-driven — expires an entry on a schedule regardless of how hot it still is; usually layered on top of LRU/LFU rather than used as the sole policy.

Random

When to use: Extremely high-throughput caches where even LRU/LFU's O(1) bookkeeping overhead is worth avoiding.

Trade-offs: Cheapest possible eviction — no access metadata to maintain at all — but with zero access-pattern awareness, hit rate is whatever the workload happens to give you.

Naive modulo hashing

When to use: A fixed, never-changing node count — the trivial case.

Trade-offs: `hash(key) % N` is trivial to implement, but changing N — adding or removing even one node — remaps nearly every key.

Consistent hashing (hash ring)

When to use: Node counts that change over time and where minimizing remap churn on each change actually matters.

Trade-offs: Nodes and keys both map onto a ring; adding or removing one node only remaps the keys between it and its ring neighbor — roughly 1/N of all keys, not nearly all of them.

Consistent hashing + virtual nodes

When to use: The same elastic-node-set problem, but with uneven physical node capacity.

Trade-offs: Each physical node gets many points on the ring, smoothing load distribution when capacity is uneven and reducing the odds of one node inheriting a disproportionate share on rebalance.

Data lake

When to use: Raw or unstructured data landing cheaply via object storage, with schema imposed later at read time.

Trade-offs: Cheap and flexible, schema-on-read — but ungoverned, it degrades into a "data swamp" nobody trusts.

Data warehouse

When to use: Fast, structured BI/analytics queries against a fixed, enforced schema — schema-on-write.

Trade-offs: Fast, well-governed structured queries, but every new source needs upfront modeling before it's usable.

Data lakehouse / Delta Lake

When to use: Wanting warehouse-style guarantees without the separate lake-to-warehouse ETL hop.

Trade-offs: Adds warehouse-style ACID transactions and schema enforcement directly on top of lake object storage, collapsing that ETL hop — but the tooling is younger and still maturing.

Data mesh

When to use: The bottleneck is organizational, not technical — a single central team owning one warehouse is the actual constraint.

Trade-offs: Domain teams own and publish their own data as products instead of a central team owning one warehouse — but this only works with real platform/tooling investment behind it, or it just relabels the same bottleneck under a new name.

First Normal Form (1NF)

When to use: The baseline for any relational table — every cell holds a single atomic value, no repeating groups.

Trade-offs: Necessary starting point, but doesn't remove redundancy on its own — that's what the next forms do.

Second Normal Form (2NF)

When to use: Tables with a composite primary key, where a non-key column only actually depends on part of it.

Trade-offs: Removes partial-dependency redundancy by splitting the table, but doesn't yet catch redundancy between non-key columns.

Third Normal Form (3NF)

When to use: The practical default target for most OLTP schemas — no non-key column depends on another non-key column.

Trade-offs: Removes transitive-dependency redundancy; most production schemas stop here since it's the best trade of integrity vs. query complexity.

Boyce-Codd Normal Form (BCNF)

When to use: Tables with multiple overlapping candidate keys, where 3NF alone still allows an anomaly.

Trade-offs: The strictest common form — every determinant is a candidate key — but stricter normalization means more joins to reassemble a full record.

Deliberate denormalization

When to use: Read-heavy paths where the extra joins a normalized schema requires are the actual bottleneck (see the Scaling progression question).

Trade-offs: Trades write-time integrity risk and storage duplication for fewer joins and faster reads — a conscious reversal of everything above, not a failure to normalize.

Object storage (S3-style)

When to use: Large, infrequently-modified blobs — images, video, backups, data-lake files — accessed by key, not by path.

Trade-offs: Durable (S3-class services advertise ~99.999999999% — "11 nines" — object durability via automatic cross-AZ replication) and effectively infinitely scalable at low cost, but no partial-file edits and higher per-request latency than block storage.

Block storage (EBS-style)

When to use: Low-latency, random-access disk for a database or any workload needing filesystem semantics on one instance.

Trade-offs: Fast, but tied to a single attached instance at a time — not natively shareable across a fleet.

File storage (EFS/NFS-style)

When to use: A shared, POSIX-compliant filesystem multiple instances need to read/write concurrently.

Trade-offs: Simplifies sharing, but adds network-filesystem latency and its own scaling ceiling compared to block storage.

Functional / vertical partitioning

When to use: Splitting by responsibility — a users table on one database, orders on another — usually the first, cheapest lever before any horizontal split.

Trade-offs: Simple and requires no new distributed-systems infrastructure, but has a hard ceiling: it doesn't help once one functional area itself outgrows a single node.

Horizontal sharding (range/hash/directory)

When to use: One functional area's write volume has outgrown a single primary — the next lever once vertical partitioning is exhausted.

Trade-offs: Real horizontal write scaling, at the cost of cross-shard queries and transactions becoming much harder — see the Reference Grid's Sharding strategies entry for the range vs hash vs directory trade-off.

Directory-based partitioning

When to use: Shard assignment needs to change without re-hashing everything — a lookup service maps keys to shards instead of a fixed formula.

Trade-offs: Most flexible for rebalancing, but the lookup service itself becomes a new dependency and potential bottleneck.

Pull-based consumption

When to use: The consumer monitors the queue itself and pulls a new message only when it has spare capacity — Kafka-style consumers are the classic example.

Trade-offs: Keeps the consumer in control of its own load, so it's never overwhelmed, but an empty queue with a slow polling interval adds latency before a waiting message gets picked up.

Push-based delivery

When to use: The queue delivers a message to the consumer as soon as one is available — SQS long-polling and RabbitMQ delivery both work this way.

Trade-offs: Lower latency between message arrival and processing, but a burst of incoming messages can push faster than a fixed consumer fleet can keep up, risking overload.

Point-to-point queue

When to use: A single logical operation with one owner — e.g. exactly one worker fleet processing payment jobs from a queue.

Trade-offs: Simple and load-balances naturally across a consumer fleet, but doesn't fan an event out to multiple independent interested parties.

Publish/subscribe (topic-based)

When to use: One event needs to reach several independent subscribers who don't need to know about each other — placing an order might need to update inventory, trigger billing, and notify shipping, all from one published event.

Trade-offs: New subscribers can be added to a topic without changing the publisher at all, which is the model's biggest strength — but message ordering across independent subscribers isn't guaranteed unless the broker specifically supports it.

MapReduce-style batch (Hadoop/Spark)

When to use: Mature, well-understood tooling and a simple map -> shuffle -> reduce mental model.

Trade-offs: Output freshness is bounded by the run schedule, and a full-dataset job pays a roughly fixed cost every run — even for a small incremental delta.

Stream processing (Kafka Streams, Flink)

When to use: Near-real-time output is a real requirement, not a nice-to-have.

Trade-offs: Real operational complexity: state management, exactly-once semantics, backpressure, handling out-of-order events.

Micro-batch (Spark Structured Streaming)

When to use: Wanting most of streaming's simplicity of reasoning without committing to a full stream processor.

Trade-offs: Splits the difference, but is still fundamentally bounded by its micro-batch interval.

Offline metrics

When to use: Gating the ship/no-ship decision — the last checkpoint before a model reaches real users, measured against a held-out test set.

Trade-offs: Cheap, fast, and reproducible, but measures fit to historical data — not what happens once the model's own predictions start changing user behavior.

Online metrics

When to use: The real, business-facing measure of whether the model helped — what the ML deployment risk ladder above is actually gating for.

Trade-offs: Measures true impact, including feedback loops offline metrics can't see, but takes real traffic, real time, and a live experiment to get a read.

Both, in sequence

When to use: The standard approach for any production model — offline gates deployment, online then validates it actually worked.

Trade-offs: Offline metrics can look great while online metrics disagree (the "offline/online gap") — a model that improved held-out accuracy can still lose on the metric that actually matters live.

Batch processing

When to use: Nightly feature computation, model retraining, large historical aggregations.

Trade-offs: Simpler to reason about and debug, but output is only as fresh as the last batch run.

Streaming processing

When to use: Real-time fraud scoring, live recommendation updates, anomaly detection.

Trade-offs: Low-latency output, but adds real operational complexity (stream processors, exactly-once semantics, backpressure handling).

Online (real-time) prediction

When to use: Fraud detection at checkout, live ranking, anything needing request-time context the model hasn’t seen before.

Trade-offs: Always fresh and context-aware, but must meet a tight latency SLA on every request and needs a always-on serving infrastructure.

Batch (offline) prediction

When to use: Daily churn scores, weekly recommendation refreshes — anywhere precomputing for all known entities is feasible.

Trade-offs: Cheap and simple to serve (just a lookup), but predictions are stale between batch runs and can’t react to request-time context.

Monitor for data drift

When to use: Detect when incoming feature distributions diverge from training-time distributions.

Trade-offs: Catches upstream data pipeline changes and population shifts early, but a drift alert alone doesn’t tell you if it’s actually hurting predictions.

Monitor for concept drift

When to use: Detect when the model’s predictions are becoming less accurate against ground truth, even if input distributions look stable.

Trade-offs: Directly measures what matters (accuracy), but requires timely ground-truth labels, which aren’t always available quickly.

Scheduled retraining against staleness

When to use: A baseline safeguard regardless of whether drift is detected — the world moves even without a dramatic shift.

Trade-offs: Simple and predictable, but retraining on a fixed schedule can be wasteful if nothing has actually changed, or too infrequent if it has.

Shadow deployment

When to use: First step for any new model — run it alongside the current model on real traffic, log its predictions, but never serve them.

Trade-offs: Zero user-facing risk, but doesn’t validate real-world behavioral impact (e.g. how users react to different recommendations).

Canary rollout

When to use: After shadow validation looks good — serve the new model to a small percentage of real traffic.

Trade-offs: Bounds the blast radius of a bad model, but needs strong monitoring and a fast rollback path to be safe.

A/B test

When to use: Once safety is established and the question shifts to "is this actually better," measured against a real business metric.

Trade-offs: Gives a statistically grounded answer on impact, but takes longer to reach significance and needs careful experiment design to avoid contamination.

WebSockets

When to use: Client-facing, real-time, bidirectional — chat, collaborative editing, anything needing client→server AND server→client.

Trade-offs: Stateful connection; needs a pub/sub backplane (Redis, NATS) to broadcast across horizontally-scaled server nodes.

Server-Sent Events (SSE)

When to use: Client-facing, real-time, server→client only — live feeds, notifications, progress updates.

Trade-offs: Simpler than WebSockets, HTTP-native, auto-reconnect built in — but one-directional only.

Long polling

When to use: Real-time delivery under compatibility constraints — legacy clients or restrictive networks/firewalls that block persistent connections.

Trade-offs: Highest latency and server overhead of the three, but the most broadly compatible.

REST or GraphQL

When to use: Batch-acceptable, client-facing — standard CRUD/resource access where staleness of seconds-to-minutes is fine.

Trade-offs: REST is simple and cacheable; GraphQL avoids over/under-fetching at the cost of query complexity.

gRPC over REST

When to use: Internal, performance-sensitive service-to-service calls where latency and strongly-typed contracts matter.

Trade-offs: HTTP/2 multiplexing and binary protobuf payloads beat REST/JSON on latency and payload size, but it's not browser-native and needs a gateway/BFF translation layer for client-facing use.

Synchronous (request/response)

When to use: The caller genuinely needs the result before it can proceed — reading a user profile before rendering a page.

Trade-offs: Simple to reason about, but the caller is blocked (and its resources tied up) for the full duration of the downstream call.

Async via queue/message broker

When to use: Fire-and-continue work — sending an email, processing an uploaded video, updating a search index.

Trade-offs: Decouples caller from callee and smooths traffic spikes, but adds eventual-consistency and requires the caller to handle "not done yet" states.

Async via callback/webhook

When to use: Long-running third-party operations (payment processing, batch jobs) where polling would be wasteful.

Trade-offs: Efficient — no polling — but requires the caller to expose a reachable endpoint and handle the callback arriving out of order or more than once.

Traditional server

When to use: Steady load, long-lived connections, in-memory caching between requests.

Trade-offs: You manage scaling, and idle capacity costs money even at zero traffic.

Serverless / FaaS

When to use: Spiky/unpredictable load, event-driven workloads, wanting zero idle cost.

Trade-offs: Cold starts, execution time limits, no in-memory state between invocations.

Stateless

When to use: Any request can be served by any instance because all needed state lives externally (DB, cache, token).

Trade-offs: Trivially horizontally scalable and resilient to instance failure, but every piece of state needs an external home, which can add latency.

Stateful

When to use: In-memory game state, WebSocket connections, or anything where re-fetching state on every request would be too slow.

Trade-offs: Faster access to local state, but requires sticky routing and losing an instance means losing (or needing to recover) its state.

Optimize for latency

When to use: Interactive, user-facing requests — search-as-you-type, page loads, anything with a human waiting.

Trade-offs: Fast individual responses, often at the cost of lower overall throughput (less batching, less resource sharing).

Optimize for throughput

When to use: Batch processing, bulk ingestion, background jobs — nobody is watching a single request complete.

Trade-offs: Batching and queuing maximize total work done per second, but individual items wait longer before being processed.

Live / on-demand computation

When to use: An unbounded or highly dynamic input space, or a result that depends on request-time context that can’t be known in advance.

Trade-offs: Always fresh, but every request pays the full compute and latency cost.

Precomputed / materialized

When to use: A bounded, enumerable input space, or an expensive computation worth amortizing — nightly aggregates, denormalized read models.

Trade-offs: Near-instant reads, but staleness is bounded by the last precompute run.

Hybrid: compute-then-cache

When to use: A large but skewed/long-tail input space, where a small hot set dominates traffic.

Trade-offs: Captures most of the benefit of both, but needs a real invalidation strategy or it silently serves stale results forever.

Token bucket

When to use: Need to allow short bursts up to a cap while still enforcing a steady average rate — the default choice for most public APIs (variants of this back AWS and Stripe rate limits).

Trade-offs: Cheap to check (compare token count, decrement), naturally burst-tolerant — but tuning bucket size and refill rate to match real traffic shape takes iteration, not a formula.

Leaky bucket

When to use: Need to smooth bursty input into a strictly constant output rate — protecting a fixed-capacity downstream (a worker pool, a legacy system) that can't absorb any burst at all.

Trade-offs: Guarantees a constant outflow regardless of how input arrives, but excess requests queue or drop rather than passing through — under sustained overload the queue itself becomes a memory/latency liability.

Fixed window counter

When to use: Simplicity matters more than precision — a quick global cap like "1000 requests per minute" behind a single counter.

Trade-offs: Trivial to implement (an atomic INCR + TTL in Redis), but allows up to ~2x the stated limit right at the window boundary — a burst at 11:59:59 and another at 12:00:00 both pass unpenalized.

Sliding window counter

When to use: Need the fixed window's simplicity without its boundary-burst flaw — most production rate limiters (Cloudflare, common API-gateway defaults) land here as the practical middle ground.

Trade-offs: Weights the previous window's count by how much of it still overlaps the current window, which closes the boundary gap at a fraction of the memory cost of a true sliding log (which stores every request timestamp for exact precision) — an approximation that's accurate enough for almost every real use case, not an exact count.

URI versioning (`/v1/users`)

When to use: Public APIs and anything where discoverability and cacheability matter most — the version is visible in every log line, browser address bar, and CDN cache key.

Trade-offs: Simplest to implement, test, and cache (each version is a genuinely distinct URL) — but it puts an implementation detail in the resource path, so "the same resource" now has multiple canonical URLs.

Header versioning (`Accept: application/vnd.app.v1+json`)

When to use: APIs that want the URL to stay a pure resource identifier and treat the version as metadata about the representation, not the resource's identity — common in stricter REST implementations.

Trade-offs: Keeps URLs clean and semantically correct, but is harder to test from a browser, less discoverable to a new consumer skimming the docs, and CDNs need explicit config to vary the cache by header instead of defaulting to URL.

Query-parameter versioning (`/users?version=1`)

When to use: A middle ground — version stays visible and easy to test in a browser like URI versioning, without minting a whole new path segment or route table per version.

Trade-offs: Easy to default (omit the param) and easy to test, but query params conventionally mean filtering/sorting, not identity — mixing the two is a recurring source of confusion, and caching by query string is less consistently supported than caching by path.

Server Component (the App Router default)

When to use: Static or per-request server-rendered content with no client interactivity — e.g. most of this page’s own text sections.

Trade-offs: Ships zero client JS for that component, but no hooks/state/browser APIs are available inside it.

Client Component ('use client')

When to use: Needs state, effects, or a browser API — e.g. this site’s own useActiveSection scroll-spy hook, which is exactly why components like SweCompassSection.tsx must be client components.

Trade-offs: Ships JS and pays a hydration cost, so the boundary should be drawn at the smallest leaf component that actually needs it, not the whole page.

Static generation / ISR

When to use: Content is identical for every visitor with no personalization.

Trade-offs: Fastest possible response, but freshness is bounded by the last build or revalidation.

Self-hosted / on-prem hardware

When to use: Strict data-residency/compliance requirements, extremely high sustained/predictable load where owning hardware amortizes below cloud rental, or an existing ops team with deep infra expertise.

Trade-offs: Full control, no vendor markup at scale, but capital cost, your team owns every failure mode, and no elastic burst capacity.

Native cloud (AWS/GCP/Azure)

When to use: Need the broadest managed-service catalog, multi-region reach, and elastic scaling without owning hardware.

Trade-offs: Huge surface area and flexibility, but real operational complexity (IAM, networking, cost governance) and a steeper learning curve.

Third-party PaaS/edge vendor (Vercel, Cloudflare)

When to use: Frontend-heavy or edge-latency-sensitive workloads, small teams that want to ship without an ops function at all.

Trade-offs: Fastest time-to-ship and near-zero ops overhead, but less control, potential vendor lock-in, and usage-based pricing that can surprise at scale.

Optimize for compute-bound

When to use: CPU is the pegged resource — heavy computation, encoding, ML inference on dense math.

Trade-offs: More/faster cores and parallelization help directly; but Amdahl’s law limits how much parallelizing a sequential portion actually buys.

Optimize for memory-bound

When to use: CPU is idle waiting on data — large working sets, cache misses, heavy data shuffling.

Trade-offs: More RAM, better caching, and reducing data movement (columnar formats, batching) help; adding CPU cores alone won’t.

CI/CD gates + infra monitoring (logs/metrics/traces)

When to use: Deterministic services where correctness is binary and failures are loud — tests and validation catch regressions pre-deploy, dashboards and alerts catch outages post-deploy.

Trade-offs: Mature, well-understood tooling and fast MTTR for hard failures; blind to silent correctness decay, which is exactly the failure mode ML components are prone to.

SLOs + error budgets

When to use: Need one shared health contract that both engineering and product hold each other to, with an explicit, pre-agreed answer to how much risk can be spent shipping this week.

Trade-offs: Forces prioritization discipline; only works if the org actually halts shipping when the budget's blown, not just on paper.

Model/output observability (drift + quality monitoring)

When to use: Any component serving a model — infra can look perfectly healthy while prediction quality degrades under distribution shift.

Trade-offs: Catches what infra monitoring can't, but usually needs labeled ground truth or a proxy metric that lags reality — see #drift-vs-staleness for the staleness/detection-lag trade-off itself.