Wanderland

Wanderland Testing — AI Primer

Companion to [wanderland-dev-ai-primer]. Read this before writing any test in a wanderland-* repo. Verified against on-disk wanderland-core, foxfire, and wanderland-looking-glass (2026-07).

Mantra: Burn potential through a boundary, expect the ashes. A boundary is a shape→shape transform. A test feeds one concrete input (the potential) and asserts the exact output (the ashes). Do this for every boundary — happy path plus every halt — then run as many full request→response cycles as the surface allows. The data-structured YAML scenario is the critical majority. Reach for an rspec integration spec only for what doesn't fall neatly into a YAML scenario.

What goes where — pick the lens

Test the… Use Because
Transform of one boundary (input shape → output shape, each halt) Scenario YAML operation: <boundary> Declarative, reviewable, the majority. Direct Boundary.execute.
A mid-chain boundary that reads upstream output Scenario YAML + input.context: seed context: simulates the crossings an earlier slot would have appended.
A boundary that calls an adapter (HTTP/shell/AWS) Scenario YAML + mocks: (or fixture-site mode: mock) Canned responses; no real network/process.
A full request→response cycle through the engine (route chain, seeds, storage writes) Scenario YAML operation: runtime_inspect + input.dispatch The single inspection seam: boot site, dispatch, assert dispatch_crossing + storage.
Signature/HMAC verification, raw request body, live round-trips, a direct-adapter.run branch rspec integration spec Needs computed crypto, raw bytes, or a spy the YAML lenses can't express.
Adapter polymorphism (same seam, CLI vs HTTP vs rack) DispatchScenario layers (wanderland-core internal) Layer 1 seam / 2 rack_test / 3 cli binary.

The split is not optional discipline vs convenience — it's capability. Scenario#run and runtime_inspect's dispatch pass params + headers only; neither supplies a raw request body. So anything verifying an HMAC over the exact bytes (Slack v0:<ts>:<raw>, GitHub X-Hub-Signature-256) must be an rspec spec that hands Boundary.execute a hand-built input with body: set, or drives the engine via Rack::MockRequest with a real signed body.

Lens 1 — per-boundary scenario YAML

operation: <boundary> dispatches Wanderland::Boundary.execute(operation, input); expected: is ShapeMatched against crossing["result"].

name: "route_hook: a github PR synchronize routes to the worker"
description: |
  A normalized delivery (what verify_hook produces) is seeded into context.
  route_hook matches it against hook_routes and resolves the forward decision.
operation: route_hook
input:
  context:                       # simulate the upstream slot (see Context seeding)
    - boundary: verify_hook
      result:
        provider: github
        kind: pull_request
        action: synchronize
        verified: true
expected:
  upstream: worker
  target: /webhooks/github/pr
  ack: 202

Files live at spec/scenarios/<boundary>/NN_name.yml; the loader (Wanderland::Scenario.load_all(dir)) sorts by filename. One assertion of intent per file. Halt scenarios assert the halt payload:

expected:
  status: 401
  error: "signature verification failed"

crossing["result"] projects the halt's payload but strips the type address — assert status/error, never _type_addr.

Lens 2 — full-cycle via runtime_inspect

The single diagnostic seam for request→response cycles. Extend runtime_inspect rather than adding a new inspection boundary — the point is uniform scenarios.

name: "cycle: PR delivery forwards to the worker and records a delivery"
operation: runtime_inspect
input:
  site: spec/fixtures/shruti_site          # boots this config.yml (mocks baked in)
  env:                                       # override the ENV snapshot for this boot
    VIVEKA_BASE: http://viveka.test
  dispatch:
    method: POST
    path: /webhooks/github/pr                # a route whose chain skips raw-body verify
    params: { provider: github, kind: pull_request, action: synchronize }
    headers: { }
  query_storage:                             # assert what landed in a store
    - prefix: ":deliveries:"
expected:
  dispatch_crossing:
    type_addr: ":types:ok"
    result: { forwarded: true, upstream: worker, upstream_status: 202 }
  storage:
    ":deliveries:": { count: 1 }

runtime_inspect inputs: site, config_file, env (ENV-snapshot override), dispatch: {method, path, params, headers}, forge (inject a crossing into the request context before dispatch), query_storage. Outputs: dispatch_crossing, request_context, boot_context, context_seeds, storage, route, user_chain, compiled_chain, registered_injections, error. Because dispatch carries no body, a cycle that must pass a body-signature check belongs in rspec.

Lens 3 — rspec integration (the "doesn't fit" cases)

Boot a fresh runtime, hand Boundary.execute an input hash you control, compute crypto live. The canonical example is foxfire's signed_ingest_spec.rb:

def dispatch(env)
  input = { "runtime" => runtime, "config" => { "keys" => { key_id => public_pem } }, "params" => env }
  Wanderland::Boundary.execute(:ingest_signed, input)["result"]
end

For raw-body signature boundaries, set body: directly and compute the HMAC in the spec:

raw = "payload=" + CGI.escape(payload_json)
ts  = Time.now.to_i.to_s
sig = "v0=" + OpenSSL::HMAC.hexdigest("SHA256", secret, "v0:#{ts}:#{raw}")
input = { "runtime" => rt, "config" => { "providers" => { "slack" => { "signing_secret" => secret } } },
          "args" => { "provider" => "slack" }, "body" => raw,
          "headers" => { "X-Slack-Signature" => sig, "X-Slack-Request-Timestamp" => ts } }
result = Wanderland::Boundary.execute(:verify_hook, input)["result"]

For a full inbound→outbound cycle with a real signed body, drive runtime.engine through Rack::MockRequest (see wanderland-core engine_body_spec.rb), and to assert the exact outbound request, swap the upstream adapter for a recording fake on runtime.adapters before the call.

rspec setup mirrors foxfire/looking-glass: a before(:each) that resets and boots a fresh runtime —

Wanderland::Formatters.reset! if defined?(Wanderland::Formatters)
Wanderland::Boundary.reset!; Wanderland::Boundary.reload!
@runtime = Wanderland.boot(fixture_config)

A scenario runner spec then iterates Wanderland::Scenario.load_all(dir) and asserts scenario.verify is empty.

Context seeding — simulate an upstream slot

To test a boundary that reads what an earlier chain slot appended, input.context: must be an array of crossing events (hydrated via raw_append, bypassing PKI):

input:
  context:
    - boundary: route_hook
      result: { upstream: worker, target: /webhooks/github/pr, ack: 202, payload: { number: 167 } }
      # optional: from_addr, type_addr (default :types:ok), signature, at

Projection rules a boundary then reads:

Mocks — canned adapter responses

Two ways, same envelope:

Per-scenario mocks: keyed by adapter name → list of {request:, response:}:

mocks:
  worker:
    - request:  { method: POST, url: "http://viveka.test/webhooks/github/pr" }
      response: { status: 202, body: '{"ok":true}' }

Only adapters that support :mock mode (implement register_mock_envelope) accept mocks; a bare data boundary can't be "mocked". Registered before dispatch, cleared after.

Baked into a fixture site (adapter mode: mock + mocks:) — the looking-glass pattern. Happy and failure paths coexist on one site by using distinct routes/args → distinct mock keys:

adapters:
  jenkins: { kind: http, mode: mock, mocks: { "GET https://…/job/demo/api/json?tree=…": { status: 200, body: "…" } } }

Mock-key rules (bite marks): HTTP key is "METHOD url" — body and headers are not in the key; a build POST reports its queue item via a Location response header. Shell key is the args: array. Miss raises CassetteMissError (loud). !Fixture <path> lifts a file as the response body; the same {request:, response:} shape works in cassettes.

ShapeMatcher — asserting the ashes

expected: vocabulary (same in scenarios and --wanderland-verify):

Form Meaning
scalar exact ==
hash subset match (extra keys OK)
count: / gte: lte: gt: lt: collection size / numeric guards
`empty: true false`
includes: [...] / excludes: [...] / contains: x membership (all / none / one)
first: / last: / any: project a collection element, match a sub-shape
matches: <regex> regex on a string value
has_key: k / keys: [...] hash structure (keys: is the exact sorted set)
not: <shape> negation

Reserved-key collision (gotcha): count first last any keys empty not matches contains includes excludes gt gte lt lte has_key are matcher keys. If a real output field is literally named one of these, ShapeMatcher reads it as a matcher, not a field. Avoid those field names; if stuck, assert via blanks: dot-paths (which dig, not match) or restructure the expected block.

Gotchas banked (each cost a cycle)

Red → green (the discipline)

For any wanderland-core change and any new boundary: write the failing case first, run it, confirm it fails for the right reason (the new kwarg/method/shape — not a setup typo), make the minimal change to green, re-run the file and the surrounding suite. Commit the test in the same diff. "Or it doesn't count."

Running

bundle exec wanderland --type test config.yml                 # all scenarios: block
bundle exec wanderland --type test config.yml --route deploy   # one route
bundle exec wanderland --type test config.yml --scenario happy # one case
curl -H 'X-Wanderland-Verify: <case>' http://…/route           # probe lens (live)
bundle exec wanderland --type cli config.yml <route> --wanderland-verify <case>
bundle exec rspec                                              # rspec units + scenario runners

Exit: 0 all pass, 1 fail/no-match, 2 bad argv. :live-tagged specs excluded unless WANDERLAND_LIVE=1.

Pointers