The markdown-graph engine — the surface the oculus_* MCP tools, [lantern], and the doom/nvim plugins drive — built as wanderland-core boundaries over the append-only overlay store that ships in core (Activities::Overlay, SQLiteDriver#overlay). Loom was the original intermediate system between Oculus and Wanderland; the name returns here.
This node fixes the shared model. It is the design that gets ported into the new loom graph as one of its first nodes. Pairs with [wanderland-core-overlay-operators] (the persistence substrate) and [oculus-api-primer] (the surface it reproduces). Build is tracked in case task-509adb4b.
As long as we have a type and an address, we are done. Everything is a typed record at an address. A whole node is the records under its address. Reads are overlays — seed plus operations composed in at order. Writes are splices — append an operation record. Differentiation is by type_addr, and many types live at the same address.
A slug's source of truth is its markdown. On ingest, a node remark bridge returns mdast — the unified/remark AST. That token stream is the seed record for the node. mdast block nodes (the root.children array) are the tokens. A section, a fence, or a virtual-fence owns a contiguous range of that stream and is lifted into its own record. Where it was lifted from, the parent stream keeps a hole — a marker that is a ref to the lifted record's address.
A node's stream, after lifting, is prose tokens plus holes. To read it you fill the holes: resolve each ref (a read, or — when the hole is a callable fence — an execute), splice the emitted tokens back, and stringify. Everything on top of a node is a ref to another pointer.
Slug widget-service. Every pack scenario uses this exact input.
# Widget Service
A small service that vends widgets.
## Configuration
Reads its config from the fence below.
```yaml[label=service-config]
port: 8080
replicas: 3
```
## Tasks
- [ ] wire the health check
- [x] add the readiness probe
- [ ] document the API
## Deploy
```python[label=deploy,execute=true]
print("deploying widgets")
```
It exercises every shape: an h1 node title plus intro prose, plain section prose, a labelled data fence (yaml), a GFM task list (the todo virtual-fence), and a labelled executable code fence (python). remark emits the fence info string whole into code.lang ("yaml[label=service-config]"); parsing type[attrs] into fence_type plus attributes is the fence-marker step's job, so the AST step is a faithful passthrough.
A hole is a marker spliced into a parent stream where a child was lifted out:
{ type: "loom:hole", kind: section, ref: ":nodes:widget-service:configuration", label: "Configuration", depth: 2 }
kind is section, fence, or vfence. ref is the address of the lifted record. After ingest the node's own stream is its title, its intro prose, then one hole per section. A section record holds its own sub-stream (prose plus holes for its fences, vfences, and nested subsections). Holes nest arbitrarily; composition is recursive self-emission — grab a slug, fill its holes, each fill grabs its own and fills theirs.
All at addresses under :nodes:<slug>. type_addr differentiates; labels can overlap across types.
- type_addr: ":loom:node" # node stream with holes
- type_addr: ":loom:section" # heading + body, owns a token range
- type_addr: ":loom:fence" # canonical fence, owns code/tokens
- type_addr: ":loom:fence-label" # label pointer → canonical fence
- type_addr: ":loom:fence-meta" # fence metadata (tags/execute/attrs), poke-updated
- type_addr: ":loom:vfence" # virtual fence (todo, table, ...), owns a token range
- type_addr: ":loom:edge" # a cardinal edge { direction, links }
- type_addr: ":loom:tag" # generic metadata on any object
- type_addr: ":loom:poke" # one edit on top of a baseline (trace → baseline)
- type_addr: ":loom:baseline-section" # a re-serialized section snapshot
- type_addr: ":loom:baseline-node" # a squashed whole-node snapshot
- type_addr: ":loom:output" # a fence execution result (input_ref + result)
- type_addr: ":loom:tombstone" # archive marker
The canonical fence is the section-fence record — it owns the code. Labels are branch pointers (:loom:fence-label) that point at it; many labels can point at one canonical fence, and revisions mount against the canonical. Fence metadata is its own record, mutated within its payload by poke.
":nodes:widget-service" # the node (stream with holes)
":nodes:widget-service:configuration" # a section
":nodes:widget-service:configuration:fs-1234abcd" # the canonical fence (opaque stable id)
":nodes:widget-service:configuration:service-config" # a label pointer → fs-1234abcd
":nodes:widget-service:tasks:todo" # the todo virtual-fence
":nodes:widget-service:deploy:deploy" # a label pointer → deploy fs-9877fedc
":nodes:widget-service:edge:east" # an edge record
":nodes:widget-service:tag" # the tag record
Querying a type under an address is the universal read: all :loom:section under a node is the heading tree; all :loom:fence under it are its fences; one address plus :loom:fence is that fence. peek resolves any of these, and the type at the address decides how it renders — routed by format through the printers lantern already uses.
The section is the atom. A section's current value is two queries: find the latest baseline (:loom:baseline-section, or the original :loom:section seed), then replay every :loom:poke at that address in at order on top, each poke's trace correlating it to the baseline it edits. This is Activities::Overlay.compose(seed, ops) — $set` / `$merge / $append on section-addressed paths, which are commutative and need no operational transform; positional splice is reserved for where it is unavoidable.
:loom:poke at the section address.A boundary chain, each step adding its breakdown to the trace and passing context forward:
1. ast # markdown → mdast (remark shell-out, faithful passthrough)
2. section_markers # mdast → section records owning token ranges; node stream gets section holes
3. fence_markers # sections → canonical fences + label pointers; section streams get fence holes
4. pattern_match # AST pattern matcher (parallel buckets, not regex) → vfence records; vfence holes
5. write_records # the decomposed node → append every typed record to the store
The trace carries all breakdowns; the records are the persisted result. Reprocess and rebuild-missing are a query plus a loop, runnable as a timed job. Virtual fences detect during this pass: a vfence owns its token range, emits a marker, and carries its own route; on read it emits its data back as a template.
- fence_info_syntax: "type[attrs] split in fence_markers (leaning) vs normalized in ast"
- fence_metadata: "one :loom:fence-meta payload, poke-updated (leaning) vs per-key sub-records"
- baseline_causal_pointer: "address + at only (leaning) vs explicit prev link"
- edge_records: "one :loom:edge per direction with links[] (leaning) vs one record per link"
- hole_node_type: "loom:hole with kind (leaning) vs distinct section-hole / fence-hole types"
Supersedes the open-decisions list above.
Holes are peek calls. A hole is { type: loom:hole, ref: <address> } — nothing more. Filling it is a peek to that address with rendering parameters (raw tokens, resolved, or with extras turned on). Reading a node is a recursive sequence of peek calls; there is no difference between filling a hole and retrieving the target directly. The target address and its type are responsible for how it renders. Execute is a rendering level — execute-bit fences run when the requested level resolves them. Call-time injections are possible, gated by scope attenuation. No per-kind hole node types.
Fence attributes are a boundary. Parsing the type[attrs] info string is a dedicated fence_attrs boundary — late-parsed, callable from anywhere. fence_markers calls it. It parses the attributes and writes the :loom:fence-meta record (parsed attrs as payload, at the fence's address); edits to fence metadata are pokes to that payload. The canonical :loom:fence record holds fence_type + the code value.
Fence identity is opaque, assigned once. A fence gets an opaque id at creation; everything references it regardless of structural position. Moving a fence between sections changes only which section's stream holds the hole — the fence's address is unchanged. Canonical fence lives at a node-scoped opaque address, e.g. :nodes:widget-service:fence:f-7a3c9e; labels are aliases that point at it; section holes ref it.
Sub-question — opaque vs opaque-content_hash. Recommendation: the opaque id is seeded from the fence's birth content hash (deterministic and stable forever — it is the birth content, not the current), and the current content hash is recorded as a per-revision attribute for integrity / dedup. The id stays stable across edits (satisfies "assigned once"), content-addressing benefits are kept, and scenarios become deterministic.
Edges: one record per link (:loom:edge per link, not per direction). Cheap to change later.
Baseline: timestamp + address. No prev causal pointer; baselines are addressed and typed, rolled up by timestamp (epoch precision).
Authorization (new requirement). Calls check whether the calling entity is authorized. Map the frontend caller to an identity; grant administrative permission for now; ensure there is always somebody — a default identity. The CLI adapter catches the OS user; the HTTP/frontend path needs one too. The boundary requirements / capabilities / identity contract is the gate; core's authorization + identity + idp are the pieces, so this is turning on the entities, not new machinery. Especially needed once execute matters. Scenarios will include authorized-vs-unauthorized coverage. Not blocking now; not to be forgotten.
Loom is a simpler expression of the crossings engine (crossings-architecture), specialized to the markdown graph and built as wanderland-core boundaries instead of the full compiler / scheduler / queue. The mapping:
to_addr (where it lives), from_addr (who put it there / identity), type_addr (what kind / capability), trace_addr (what caused it). Loom records are these crossings: from_addr is the calling identity (the authorization handle), trace links a poke to its baseline and an execution output to its input. wanderland-core's SQLiteDriver already stores all four plus signature and at.type or from filters the layers. Loom's single-address overlay (seed + pokes in at order) is the simple case of that collapse — "a type under an address."loom_ast shells out as the simple version, alignable to the remark bridge later; fence executors are the fence-execution bridges, simpler.from_addr prefix and role shape (role in [admin, editor], from_addr prefix :sessions:admin). That is exactly the RBAC requirement: the calling identity is checked, admin for now, a default identity always present.Loom stays the simple version — boundaries + overlay, no compiler/scheduler. The crossings engine is the fuller system this grows toward.
The canonical :loom:fence value is the source string, not parsed data. "Give me it as data" is the @L4 render level — peek parses the string on read and hands back the data (L3 = raw string, L4 = parsed data, L5 = formatted). Reasons: exact round-trip back to markdown (comments, key order, formatting survive a parse→dump that would otherwise drop them); the opaque / content-hash id wants the canonical bytes; no dual-representation desync (the two-serializer trap this rebuild escapes); and parse is a cheap pure function the overlay can memoize if it ever costs. Also the thunk-friendly choice — the string is the thunk body, so storing parsed data would pre-evaluate and kill laziness.
Fence attribute note: attrs includes tags ({ label, execute, tags: [...] }) — it's part of the :loom:fence-meta payload; tag edits are pokes to it.
Depth is preserved as authored — no promotion. The node owns everything before its first child heading (the h1 title + intro prose, h1 stays h1); each child heading is lifted to a section record keeping its authored depth, with a hole left behind. Promotion (a standalone section read rendered with its top heading as h1, descendants shifted) is a render/move concern, not storage — storage round-trips to the original markdown; emission normalizes to context.
Patterns are authored as markdown and compiled through the same loom_ast — authors never touch the AST. A pattern is a YAML file shipped with the pack:
name: todo
kind: vfence
example: | # real markdown, full content
- [ ] wire the health check
- [x] add the readiness probe
pattern: | # one representative unit, with {placeholders}
- [{checked:bool}] {text}
expected: # the data-as-map the match must produce
items:
- { checked: false, text: "wire the health check" }
- { checked: true, text: "add the readiness probe" }
Load = self-validate. Parse pattern → AST → compile {name[:type]} literals into capture markers; parse example → AST; run the compiled pattern over the example; assert the result equals expected; register on success. Every pattern carries its own test — loading is validating, and the file doubles as a scenario.
Design notes:
items[]).{checked:bool}, {count:int}, {date}); checked binds the GFM listItem.checked field, text placeholders bind text values.## Deploy {date} then repeated - {service}: {status}). Todo is the first instance; reports/changelogs/tables fall out of the same matcher.Source / boot / publishing:
lib/wanderland_loom_pack/patterns/*.yml; todo published first. The pack entry globs and registers them at load (same as boundaries + archetypes), self-validating as it goes — the boot hook.pattern_load boundary: YAML → compiled + validated pattern. pattern_match's patterns input is this registered set.loom:seed:patterns to seed/publish; a load route lets anyone drop a YAML to publish a new vfence pattern.A read never emits a flattened markdown blob. By type:
stream: the resolved token stream (holes filled recursively).data: the parsed map.value: the code string (source legitimately is a string).:loom:output record.Turning a stream into a markdown string is a separate stringify step at the display edge — the remark bridge run backwards — never the core read's output.
Boundaries never touch SQLite directly. Store access is a wanderland-core adapter (a thin client) that boundaries declare (adapters: [:store]) and call: overlay(addr), at, under, from, typed, append. It returns structured records (crossings) — the rich signaling that makes reads more than a string.
The adapter abstracts where the store lives:
runtime.storage (SQLite by :nodes: / :streams: prefix). Default, simplest.This is the crossings bridge idea applied to the store, and the same seam fence execution uses — exec_ruby / exec_python run out-of-process as execution bridges, so running code never blocks the engine either. Core's adapter machinery (recording_http / recording_shell) means a recording_store drops in, giving traceable/testable store access. Scenarios seed the store adapter's in-memory view (the store: block); boundaries read through the adapter exactly as in production.
peek takes format: default tokens returns the structured stream; markdown returns the stringified page (the remark bridge run backwards, at the display edge); data/json returns parsed data. Same read, requestable either way — structured is the default, the string is an explicit projection.
When a fence executes, its runtime is handed a scoped graph client. Inside the fence: graph.peek(addr), graph.poke(addr, …), graph.execute(addr). Those calls re-enter the same dispatch — the same peek/poke/execute boundaries every other caller uses — through the fence's own access point. The fence's scoped identity flows through the whole chain, so RBAC/denials apply uniformly, and the grant is attenuated from the caller (a fence can never out-scope whoever triggered it).
The grant carries the conditions: may-peek / may-poke / may-execute, over which address prefixes. execute_fence mints it (attenuated) when it spawns the runtime.
Transport — both tiny; lean is (1):
graph client speaks a tiny JSON-RPC over the pipe it already has to its parent bridge; the bridge fulfills each call by dispatching the same boundary with the scoped identity and pipes the record back. The fence holds no token and needs no network — scope stays server-side. Same JSON-RPC the remark bridge speaks.LOOM_ENDPOINT + a minted scoped token (env) and calls the HTTP/JSON-RPC API directly.The only per-language work is a ~10-line client shim (peek/poke/execute → JSON over the channel). Runtime binding + shim = the whole cost of a new language; dispatch, RBAC, and the store are shared. This is the same graph-access adapter from the store-access note, bound (scoped) into the runtime instead of into a boundary.
The grant adapter, the store client, and external callers all speak JSON-RPC to one endpoint. One door, scoped by the identity on the call, serving three modes off the same dispatch: one-shot fence execution, external API (editors / lantern), and an interactive session.
That third mode is a REPL. A JSON-RPC bridge into a live Ruby/Python runtime with scoped graph.peek/poke/execute is exactly a Read-Eval-Print-Loop — a fence is a one-shot eval; a REPL is the same runtime kept warm and fed lines. It's graph-bound (you do knowledge work against the store from the prompt, not bare arithmetic), fully constrained (runs under the same attenuated scoped identity, so denials/RBAC bound it), and safe to hand a human or an agent. It's bedrock made interactive: read = pause, eval + graph calls = fetch, splice the result, continue. Matches the REPL the crossings-architecture lists as pending. The grant adapter and the REPL are the same door — building the former gives the latter.