Designing a Memory-Mapped Cache Engine

1. The Design Problem

The target workload combines a high proportion of point reads with a smaller stream of puts and deletes, while the canonical data set and indexes may be larger than the process heap. A practical design must ingest at least one million typed records, validate large collection catalogs at startup without rebuilding heap-wide maps, and preserve a direct single-server data path. Because the system is a cache rather than the source of truth, it may acknowledge a bounded asynchronous suffix that can be lost after machine failure; it must never expose a partial mutation, the wrong record, a mixed file generation, or an unchecked offset.

1.1 Engineering goals

• Bound heap usage independently of the total number of records and indexed terms.
• Keep immutable record and index files directly readable through memory maps.
• Recover by validating complete frames and replaying only a bounded mutable suffix.
• Separate authoritative data from secondary structures that can be quarantined and rebuilt.
• Make backpressure, disk headroom, query ceilings, and maintenance admission explicit.
• Collect reproducible p50, p95, p99, p99.9, RSS, mapped-byte, restart, and error evidence.

2. Architecture Overview

Each collection owns a short commit sequencer, a bounded mapped recovery generation, and one or more immutable mapped generations. Requests are validated and encoded outside the sequencer. The sequenced section appends one complete recovery frame, updates the mapped primary delta, and then advances the visible sequence. Background flush advances a separate durable sequence. This split makes the asynchronous durability contract observable while keeping CPU-heavy parsing and encoding away from the serialization point.

3. Portable, Self-Validating Storage

Every file begins with a pointer-free, little-endian header and uses relative 64-bit offsets. Bounded blocks carry CRC32C checksums; authoritative manifests bind referenced complete files with SHA-256 digests. Published files are immutable and mapped read-only, and the storage layer owns the file handle for the full mapping lifetime. That ownership rule is essential on platforms where mapped-file retirement and deletion semantics differ.

3.1 Canonical records

JSON or another interchange representation is decoded against the collection schema and rewritten into a canonical record layout before commit. The layout stores stable field identifiers, fixed- and variable-width regions, document version, commit sequence, tombstone state, and absolute expiration. The record checksum is verified before a typed view is returned. Canonical bytes then become the common identity used by primary and secondary indexes.

3.2 Publication protocol

1. Write a complete new generation to a restrictive, same-directory temporary path.
2. Flush the content, finalize its checksums and digest, and rename to a never-reused generation name.
3. Create and validate the next manifest that references the complete authoritative bundle.
4. Publish the manifest and swap the in-process generation view only after validation succeeds.
5. Treat unreferenced output as an orphan that can be reclaimed; never infer authority from directory contents.

4. Mutation and Recovery

The mutable write path combines an append-only recovery log with a mapped open-addressed primary delta. A frame contains the complete key and canonical record or tombstone, which makes the delta derived state. Recovery scans to the last complete frame, ignores a torn final frame, rejects interior structural or checksum corruption, and rebuilds the primary delta idempotently.

4.1 Sealing and write backpressure

Before a mapped log, table capacity, configured load factor, or probe bound is exceeded, the active generation is sealed into immutable record, primary-index, and document-table files. When the system cannot reserve mapping capacity or disk headroom, it rejects additional writes with a stable resource error while preserving safe reads. This is preferable to silent heap growth or an unbounded maintenance backlog.

5. Primary and Secondary Query Paths

5.1 Exact-key lookup

The primary index uses seeded 64-bit hashing, linear open addressing, full canonical key verification, a published load ceiling, and a bounded probe count. The current delta is checked first, then immutable generations from newest to oldest. A tombstone terminates lookup. Stable document identifiers are never reused, and physical offsets remain local implementation details rather than distributed identities.

5.2 Equality and membership

Immutable equality indexes separate a sorted term dictionary from postings. The builder selects an encoding based on local density: inline document identifiers for very small sets, delta-varint lists for sparse sets, containerized bitmaps for clustered sets, and base-relative dense bitmaps when span density justifies them. Boolean AND evaluates smaller candidates first; OR unions; NOT subtracts from a bounded live universe. Every candidate is revalidated against the newest authoritative record, so stale secondary entries can increase work but cannot change correctness.

5.3 Range, TTL, and pagination

Numeric, decimal, date, and timestamp values use sortable byte encodings. Sealed generations store fixed-width sorted range runs, while the active generation appends bounded add/remove markers. TTL is an absolute timestamp enforced immediately on every read; cleanup later emits ordinary tombstones. Pagination uses stable document ordering and a stateless cursor bound to the collection, schema, query, projection, and last document identifier. Integrity checks reject tampered or cross-query cursors.

5.4 Correlated query batches

A server-streaming batch API accepts bounded, independently identified queries and emits chunks in completion order. Per-batch and process-wide semaphores bound parallel work. A bounded response channel propagates receiver backpressure, item deadlines include scheduling and execution, and disconnects abort or discard remaining work. Correlation never depends on arrival order.

6. Compaction Without Invalidating Readers

Append-only generations make foreground mutation inexpensive, but obsolete records, tombstones, expired values, and stale postings must eventually be removed. Full-tier compaction rewrites only the current live set into one replacement generation. Readers retain shared, pinned views during the long rewrite. Mutations pause only for the collection gate and the final view swap. A crash before manifest publication keeps the old view authoritative; a crash after publication selects the new view and later reclaims orphaned output.

6.1 Corruption domains

Authoritative record, primary, recovery, catalog, and manifest corruption fails the collection closed. Equality, posting, or range corruption can instead quarantine secondary queries while primary reads continue. An online rebuild derives new secondary structures from authoritative records and clears quarantine only after a new validated manifest is published.

7. Operations and Security

Production readiness requires more than a fast data path. Separate data and administration listeners, TLS or mTLS, bearer or certificate identities, role authorization, per-principal quotas, bounded streams, non-root container execution, checksums, audit events, and safe error messages are part of the architecture. Operational status must expose visible and durable sequence lag, disk-pressure state, secondary quarantine, pending mapping reclamation, maintenance progress, and bounded query activity without logging documents, tokens, private keys, or internal paths.

8. Qualification Evidence

The prototype was qualified through real-process tests, deterministic datasets, public gRPC drivers, crash-boundary injection, malformed fixtures, and cross-platform builds. The following retained results describe specific recorded hosts and are engineering evidence, not portable service-level guarantees.

Source note: retained qualification reports and verification documents from the independent implementation. Results are host-specific and include the configured integrity and security controls.

9. Tradeoffs and Lessons

• Memory mapping removes heap copies; it does not remove lifecycle complexity. Mapping ownership, pinned readers, and deletion retries must be first-class concerns.
• Checksums identify damaged blocks, while manifests define authority. Both are required to avoid accepting mixed generations.
• Stale secondary entries are tolerable when every candidate is checked against authoritative current state and work remains bounded.
• Asynchronous cache durability must be explicit. Separate visible and durable sequences turn a vague promise into an observable contract.
• Compaction is a resource-governance problem as much as a storage algorithm. Disk reserves, I/O pacing, heap admission, and query fairness determine production behavior.
• Benchmark reports need reproducible datasets, exact configuration, errors, percentiles, memory evidence, and restart behavior; a single throughput number is insufficient.

10. Conclusion

A production-oriented cache engine earns trust by making its failure boundaries as deliberate as its fast path. Immutable mapped generations provide a compact read representation; the bounded recovery suffix makes mutation replay predictable; adaptive indexes align representation with data density; and manifest publication turns multi-file state into one validated decision. The result is not a general-purpose database. It is a refillable cache architecture whose resource limits, loss window, corruption behavior, and maintenance contract are explicit and testable.