Wanderland Core Guide: Storage

Crossings land in storage the moment a boundary finishes. The storage engine is the substrate: a prefix-mounted driver registry, an append-only record shape, antiparticle cancellation, IOU extraction for large payloads, and a signed hash chain linking every crossing to its predecessor. On the way back out, the format slot picks a renderer by MIME and emits bytes for the adapter.

The lifecycle of a crossing

Boundary.execute returns a Signal
            │
            ↓
Walker builds a Crossing (boundary, from_addr, result, type_addr, at)
  .link(prev)    → trace = prev.signature   (hash link)
  .sign(storage) → signature over canonical payload
            │
            ↓
Context.append(crossing)
  verify signature (soft — accepts unsigned, flags forged)
  route through Storage::Registry if context has a to_addr prefix
            │
            ↓
Storage::Registry.resolve(to_addr)   → longest-prefix driver match
  driver.append(record)
    IOU.extract(payload) splits large leaves into marker + content bytes
    SQLite INSERT into records table (lean payload, sig, at)
    File/HTTP/S3 write for IOU contents, keyed by marker._args
            │
            ↓
Later: Storage::Registry.at / under / typed / overlay / hydrate

Shape of a record

to_addr     where the record belongs  (":streams:mdast:my-doc")
from_addr   who wrote it              (":sessions:claude", ":boundaries:yaml_loader")
type_addr   kind of record            (":types:poke", ":types:crossing", ":types:antiparticle")
payload     JSON — large leaves extracted as IOU markers
at          ISO-8601 timestamp
sig         signature over from_addr + canonical payload
ref         optional file pointer for blob-backed content

Everything is an address. Addresses are colon-delimited paths. The first segments determine which driver handles the read/write. The registry maps prefixes to drivers — longest match wins.

Shape of a mount table

storage:
  mounts:
    ":db:crossings":    { driver: sqlite, path: data/crossings.db }
    ":streams:mdast":   { driver: sqlite, path: data/streams.db }
    ":traces":          { driver: sqlite, path: data/traces.db }
    ":cache:snapshots": { driver: sqlite, path: data/cache.db }
    ":files:local":     { driver: file,   root: data/files }
    ":pki:":
      driver: sqlite
      path: ":memory:"
      content_drivers:
        - condition: { size: { gt: 4096 } }
          driver: file_store
          args: { root: /var/wanderland/keys }

Within a mount, content_drivers: declares when leaves extract to external backends. First matching condition wins; no match means the leaf stays inline.

Shape of an IOU marker

# Before extraction — leaf > threshold
payload:
  result:
    content: "# Very long markdown...\n\n... 12KB ..."

# After extraction — lean record, content elsewhere
payload:
  result:
    content:
      _iou:    "a1b2c3d4-..."        # unique handle
      _driver: "file_store"           # which plugin holds the bytes
      _args:   { path: "/data/blobs/a1b2c3d4.md" }   # opaque to the index
      _size:   12482                  # bytes
      _sha256: "…"                    # optional integrity hash

The SQLite index is the source of truth for what happened, when, and signed by whom. Content storage is where the bytes live. You can swap backends without losing provenance; query across all writes without touching every backend.

Antiparticles, not deletes

Nothing is deleted. Cancellation is a record with type_addr: ":types:antiparticle" whose payload references the original record's address and timestamp. On read, the overlay skips cancelled records. The antiparticle and the original both remain — the cancellation is itself a signed, timestamped fact.

to_addr:   ":streams:mdast:my-doc"
from_addr: ":sessions:claude"
type_addr: ":types:antiparticle"
payload:   { cancels: { at: "2026-04-06T03:15:00Z", type: ":types:poke" } }
at:        "2026-04-06T04:00:00Z"
sig:       "def456..."

An antiparticle must be signed by an identity authorized to cancel at that address. The check is the same capability predicate boundary authorization uses.

The hash chain

Every crossing carries two fields that make the append log tamper-evident:

• sig — signature over the crossing's canonical payload (keys sorted, JSON-serialized, signature field stripped before signing).
• trace — the sig of the crossing that caused this one.

Because trace is part of what sig covers, the parent hash is bound into this crossing's signature. Tamper with any crossing and every descendant's signature breaks; re-signing any one crossing forces re-signing every descendant, which requires every descendant's private key.

Context#verify_chain walks the stack checking two things per crossing: sig_valid (did the signature verify at append-time) and link_valid (does trace equal the previous crossing's signature). A chain is valid: true only when every crossing passes both.

Roll-up via seal

No pre-computed root hash over the whole chain. When an external system needs one — compliance export, notary handoff, cross-org attestation — the seal boundary produces it as another crossing, injected at the tail via core_injections and signed by engine:seal. The seal's signature covers the sorted list of leaf sigs plus its own trace into the linear prefix: one commitment binds the entire DAG.

Best practices

Never delete. Emit an antiparticle. The cancellation carries signed provenance; the original stays in the log. A pure delete erases evidence.

Let IOU extract large leaves. A boundary returns a hash with big strings; the storage driver walks the payload, extracts leaves over threshold (default 4KB), writes them through the configured content driver, and replaces the leaf with a marker. The SQLite row stays lean. Downstream code that needs the bytes calls hydrate; code that only needs "did this happen" skips it.

Address with prefix hierarchy. Drivers mount at :prefix:, longest match wins. Push related records together under a common prefix so under(prefix) reads them in one scan, and so mount routing stays predictable.

Write a formatter as a boundary with serves: not as a hand-rolled response shape. The serves: DSL keyword auto-registers the boundary into Wanderland::Formatters. The format slot at the tail of every chain picks the match for context["effective_mime"], executes it as a normal boundary, and stamps {body, content_type, formatter_used} on the final crossing. Return shapes stay uniform; wire format decides at the edge.

Default the MIME explicitly. Sites declare format: { default: application/json } at boot; unmatched mimes with no default halt 406. Strict mode means a missing formatter is visible, not silently coerced.

The merkle chain is soft. Unsigned crossings (no identity, signing failed, forged) still enter the context (audit trail preserved). They just show sig_valid: false through Context#verify_chain and are invisible to trust filters like .signed. Don't raise on missing keys — let the observer decide the trust threshold.

Contents

The deeper nodes for each topic in this category:

• Storage Engine — Prefix-mounted driver registry, the record shape, the address space, antiparticle cancellation, SQLite and file drivers, the registry's resolve/append/at/under/typed/overlay/hydrate surface.
• IOU Drivers — Plugin system for content backends. Typed markers (_driver, _args). Driver interface (write(content, hint:) → marker, read(args) → bytes). Per-prefix content_drivers config. Provenance survives backend migration.
• Merkle DAG — The trace chain as a linked Merkle structure. sig/trace construction. Soft signing. Context#verify_chain. The seal boundary as tail-injected root commitment.
• Format — The response-side counterpart to dispatch. Formatter registry, serves: DSL extension, the format slot at chain tail, shipped JSON/text/HTML/markdown formatters, boot-time default mime.

Site Audit

wanderland.dev