Deep technical analysis, underlying data structures, time & space complexity, production failure modes, interactive visualizers, and battle-tested Staff-level insights from Designing Data-Intensive Applications (DDIA), Database Internals, OSTEP, and modern frontier AI systems.
Visualize how keys map clockwise to the nearest virtual server token. Add or remove nodes to see that only a fraction of keys are redistributed.
Capacity = 10 tokens. Refill rate = +2 tokens/sec. Fire requests to see real-time consumption vs HTTP 429 rate limit drops.
Simulate downstream microservice calls. When failure rate reaches 50% across 6 requests, the circuit trips from CLOSED to OPEN (failing fast). After 5s cooldown, it enters HALF-OPEN canary mode.
Insert strings into an 18-bit array using 3 independent hash functions. Query items to observe guaranteed zero false negatives and occasional false positives.
| Algorithm / Family | Priority | Category | Underlying Core Data Structure | Complexity (Time & Space) | Primary Production Failure Mode | Best-Fit Architecture Scenario |
|---|
Staff engineers make trade-offs clear before choosing technologies. Use this battle-tested guide during architecture reviews and RFC discussions.
Choose Two-Phase Commit (2PC) when: Strict synchronous atomicity is required across partitions inside a single distributed database cluster (e.g. Spanner, CockroachDB) backed by consensus.
Avoid 2PC when: Crossing microservice boundaries or WAN networks; coordinator crashes hold locks indefinitely ("in-doubt"), killing system availability.
Choose Saga Orchestration (Temporal, Step Functions) when: Cross-service distributed workflows require asynchronous decoupling, human approvals, or multi-step payment flows with idempotent compensating rollbacks.
Choose MVCC (PostgreSQL, MySQL InnoDB) when: Workloads require high read concurrency where reads must never block writes and writes must never block reads.
Avoid naive MVCC without monitoring: Table bloat accumulates if long-running queries block autovacuum from reclaiming dead tuples.
Choose Serializable Snapshot Isolation (SSI) when: You need true serializability without the deadlocks and performance degradation of strict pessimistic 2PL.
Choose LSM Trees (RocksDB, Cassandra) when: Write throughput is extreme; ingest is append-heavy; SSD sequential write performance is paramount.
Avoid LSM when: Workload requires heavy random point reads or deterministic predictable low tail latency (compaction stalls can cause latency spikes).
Choose B+ Trees (PostgreSQL, InnoDB) when: Workload is read-heavy; requires in-place transactional updates with ACID compliance and efficient sequential range scans.
Choose SkipLists (RocksDB, LevelDB, ConcurrentSkipListMap) when: Highly concurrent multi-threaded writes require lock-free insertion without global tree rebalancing rotations.
Choose Red-Black Trees / AVL when: Single-threaded in-memory lookup where memory overhead of forward pointer layers must be strictly minimized.
Choose Columnar Stores (ClickHouse, Parquet, DuckDB) when: Analytical queries aggregate millions/billions of rows across a few columns (AVG, SUM); achieves 10x RLE compression and SIMD vector speed.
Avoid Columnar when: Workload requires single-row transactional point updates and writes of individual rows.
Choose Row-Oriented (Postgres, MySQL) when: Standard transactional CRUD operations fetching full row entities by primary key.
Choose Raft (etcd, CockroachDB, KRaft) when: Building a modern replicated log system where maintainability, debuggability, and strict leader invariants matter.
Avoid Raft when: You cannot tolerate traffic funneling through a single elected leader (requires Multi-Raft or leaderless Paxos).
Choose Paxos / Multi-Paxos when: Legacy compatibility with established infrastructure (Google Chubby, Spanner) or complex asymmetric geo-distributed replication topologies.
Choose TCP BBR when: Serving internet edge traffic over mobile or lossy networks; paces bandwidth at bottleneck capacity and prevents Bufferbloat.
Choose QUIC / HTTP/3 when: Mobile users experience head-of-line blocking on multiplexed streams or connection drops when roaming between Wi-Fi and 5G.
Choose HNSW when: High recall (95-99%) and sub-5ms query latency are paramount for enterprise LLM RAG pipelines and RAM budget allows ~1.5x-2x vector size.
Choose IVF-PQ when: Vector collection scales into hundreds of millions to billions, and RAM cost requires 8x-16x vector compression via product quantization.
Combine with RRF (Reciprocal Rank Fusion): Always blend vector semantic search with BM25 keyword search for robust enterprise retrieval.
Choose PagedAttention (vLLM, TensorRT-LLM) when: Serving high-concurrency LLM inference; eliminates 60-80% VRAM waste from memory fragmentation and enables dynamic prefix caching.
Pair with Speculative Decoding: Accelerates token generation by 2x to 3x using small draft models to overcome GPU memory-bandwidth latency bounds.
Choose CRDTs (Yjs, Automerge) when: Local-first, offline-capable mobile apps or peer-to-peer collaboration without round-trip latency to a central server.
Avoid naive CRDTs when: Storing long-lived documents with millions of edits without aggressive tombstone garbage collection.
Choose Centralized DB Locks / OT when: Strict global financial invariant checks (e.g. inventory cannot go negative) where total order must be authoritatively certified.