Wanderland

Walking the Grid

This is the second walkthrough in the Engine in a Box series which is the companion piece ot a series of shorts on Sprout that are themselves teaching platformm engineering concepts.

It's... a DAG

You may not know it, but you're running a DAG each time you're out at the shops with a list; a series of steps taken in order that don't repeat because if you did, you'd be late for dinner and the missus would be cross.

Today we're going to put a DAG in the form of a grid on inside of Wandlerland's engine — a small map a request walks one tick at a time — and run our errands through it. A trip to the pet store, grocer, and home. At each stop, we'll simulate a small transaction as a manual gate before proceeding.

The map's cells call into two new boundaries:

By the end of the tutorial, your sprout-engine build will answer POST /run end-to-end and POST /run/step one tick at a time, and the same shapes you exercise with curl are the shapes the puzzle UI will POST when it lands later in the series.

When you're done you'll be able to:

This tutorial picks up where Engine in a Box left off — the engine scaffolded at ~/working/sprout/sprout-engine/, port 9295, the /echo route still live for sanity checks.

What a grid route is

A standard route maps a request to a chain of boundaries that run once, in order, and produce a response. That shape fits most HTTP work — read a record, validate input, persist the change, send it back.

Some work has a different shape. The decision tree branches. Steps depend on each other but not always linearly. The whole flow is meaningful as a graph, not a list. A user might want to pause halfway through, fill in a value, and continue. An operator might want to re-route the work entirely without touching the code that does the work.

A grid route is the engine's answer to that shape. The route is still a chain — every route in the engine is — but the chain has one slot, the run_grid boundary, and the level itself rides as args.grid. Each cell in the level holds a boundary; each connection points at a next cell. Items spawn at positions named in entries:, accumulate context as they move, and are harvested at positions named in exits:. The walk is described entirely in YAML on the route's args; the boundaries it calls into are reusable across grids.

An item in the grid is a located context — a position, plus the append-only log of every crossing the boundaries it has walked past have produced. There is no separately-stored "state": "state" is just a reading of context. Bracket access (context["bag"]) projects the value via last-writer-wins; dot access (context.oculus_fence) returns the full crossing for that boundary; chainable filters (signed, for_boundary, by_identity, with_capability) narrow before resolution. See Context for the full surface. Boundaries contribute by returning a result; run_grid wraps the result in a crossing and appends it to the item's context. The next tick reads the new context.

That makes a grid route a fit for:

The shape on the page is small. Here's a three-cell grid that fetches data, transforms it, and tells the page to refresh:

routes:
  /pipeline:
    method: post
    name: pipeline
    chain:
      - boundary: run_grid
        args:
          grid:
            max_ticks: 5
            entries:
              source:
                payload: {}
            exits:
              sink: {}
            cells:
              source:
                boundary: http_get
                args: { url: "https://example.com/api/data" }
                connections: { east: transform }
              transform:
                boundary: jq
                args: { expr: ".items | map({id, name})" }
                connections: { east: sink }
              sink:
                boundary: emit_signal
                args: { action: refresh, target: results-board }

A request to /pipeline walks source → transform → sink. Three cells, three ticks. The same three boundaries can wire into any other grid; the level is the composition. Lift the grid spec into a top-level grids: block and use !Reference grids.<name> in the args to share one definition across multiple routes.

Every grid route is a chain route with one slot. That means the framework's default injections wrap a grid the same way they wrap any other route: [enforce_denials, run_grid, verify_route, trace_emit, format, seal]. verify_route runs scenarios against grid output the same way it runs them against any other route; format produces the response; seal signs the trace. There is no special grid path in the engine.

Three pieces of grid spec do the work:

Each tick, for every live item:

The run ends when no items remain in flight or ticks_remaining reaches zero. The response carries three fields:

What run_grid does

run_grid is the boundary that walks the grid. The route declares it explicitly as a chain slot and hands it the level via args.grid; injection-folding wraps the chain with the standard enforce_denials / verify_route / trace_emit / format / seal slots like any other route.

Input shape:

input:
  params: { ... }                        # request params; may seed/override entries
  args:
    grid: { entries, exits, cells, max_ticks }
    ticks_remaining: 1                   # optional; defaults to grid.max_ticks
  context: <Wanderland::Context>         # the chain context; per-cell crossings
                                         # land in a sub-stream under a fork-point

Each item is { id, position, context, awaiting?, halted? } — a located context. The item's context is a scoped view onto the request's shared event log: cells the item walked past land under :trace:req-X:N:run_grid:item-M:K, where :N is the grid slot, :run_grid is the fork-point reason, and :item-M:K namespaces per-item, per-tick. The merkle DAG threads through the fork-point: every cell links back to the previous cell in its item, every item-seed links to the fork-point, the fork-point links back to the boundary that ran before run_grid on the main chain.

Bracket access (context["bag"]) projects values via last-writer-wins inside the item's scope; dot access (context.boundary_name) returns the full crossing record; chainable filters narrow the eligible set before resolution. See Context for the full reference. Items don't carry a separate state field on the wire — sending the item back is sending the address of its context, and the engine reads what it needs from the merkle log.

run_grid ticks each item up to ticks_remaining cycles, appending every per-cell crossing to that item's sub-stream. When all items are paused, halted, or harvested, the run terminates and run_grid returns { items: [...] } — the lifecycle outcome. The response shape (value, events) comes from a downstream mapping boundary (collect_grid_state in the standard composition), which widens its scope across the grid sub-stream and projects.

Two ways an item can stop short of harvest:

What a scenario verifies

A Wanderland::Scenario is a paired input and expected output that exercises a route or a single boundary and asserts the result has the shape the scenario claims. The engine's equivalent of a unit test, written declaratively in YAML rather than imperatively in Ruby. Scenarios live alongside the code they verify, the engine picks them up at boot, and the same primitive answers three different lenses — CLI runner, HTTP introspection, and rspec.

Scenarios run in two modes:

This tutorial uses route mode: each scenario claims what /run's response should look like after a complete walk through the grid.

Anatomy

The YAML for a route-mode scenario:

scenarios:
  run:                       # route name (matches `name:` on the route)
    full_walk:               # scenario name (any identifier)
      description: "Cold-start walk completes; bag holds two purchases."
      input:
        params: { ... }      # request params for this scenario
        headers: { ... }     # optional; mock headers if the route uses them
      expected:
        items: []            # the response shape to match
        value: { bag: { count: 2 } }
        events: { count: 3 }
      mocks:                 # optional; per-scenario adapter mocks
        http: [ ... ]        # mock envelopes registered before dispatch, cleared after

How ShapeMatcher compares

Wanderland::ShapeMatcher walks two structures in parallel — the actual response on one side, the expected shape on the other — and at each position either resolves a predicate or compares for equality.

The expected shape can mix two kinds of keys:

A few examples.

Literal hash — every named field must match its expected value exactly:

expected:
  status: ok
  service: sprout-engine

A count predicate at array level — bag has exactly two items, content unspecified:

expected:
  bag: { count: 2 }

Mixed predicate and shape — bag has two items, and the first matches a shape:

expected:
  bag:
    count: 2
    first: { item: "cat food" }

in_order — the array contains these shapes in this order, gaps allowed:

expected:
  events:
    in_order:
      - { type_addr: "signals:web:oculus-refresh", payload: { section: pet-store } }
      - { type_addr: "signals:web:oculus-refresh", payload: { section: grocer } }

Numeric predicates — gte / gt / lt / lte against numeric or sized values:

expected:
  duration_ms: { lt: 500 }
  bag:         { gte: 1 }

The full predicate registrycount, gte/gt/lt/lte, matches, prefix, contains, includes, in_order, run, occurs, nth, first, last, excludes, any, keys, has_key, not, empty, all, any_of, is_a, one_of, begins, ends, min_length, max_length — is in lib/wanderland/shape_matcher.rb.

Diagnostics and usage

wanderland --type test config.yml walks every scenario the runtime knows about, dispatches each through its route, and reports pass/fail:

bundle exec wanderland --type test config.yml
# config scenarios
#   run
#     ✓ full_walk
# 1 scenario, 0 failures

A mismatch surfaces with the failing path and the disagreement, straight from ShapeMatcher:

✗ full_walk
  value.bag: array length — expected 2, got 1
  value.bag.first.item: expected "cat food", got "milk"

Filter to one route or one scenario:

bundle exec wanderland --type test config.yml --route run
bundle exec wanderland --type test config.yml --route run --scenario full_walk

Exit code is 0 on a clean pass, non-zero if anything failed. bundle exec wanderland --type test config.yml can be dropped straight into CI without a wrapper.

/inspect/scenarios returns the same registry the runner walks, so an operator can see what tests the running engine has registered without rerunning anything:

curl -s http://localhost:9295/inspect/scenarios | python3 -m json.tool

The same scenarios pin into rspec via a one-liner:

# spec/scenarios_spec.rb
require "spec_helper"
runtime = Wanderland.boot(File.join(__dir__, "..", "config.yml"))
Wanderland::Scenario.rspec_describe!(runtime)

That generates a describe per route, with an it per scenario. Failures will print the same ShapeMatcher output verbatim. bundle exec rake spec will run the set of registered scenarios in order.

For our tutorial, these scenarios are the puzzle's solved-state contract. Anything the UI claims solves the level has to satisfy verify_route — same shape match, same predicate vocabulary. The scenarios: block in config.yml becomes the authoritative answer key.

1. Add the /run route

Open ~/working/sprout/sprout-engine/config.yml and lay out the grid in two parts: a top-level grids: block that defines the level once, and route entries that reference the level by name. The two endpoints share one definition through the !Reference resolver:

service: sprout-engine
port: 9295
boundary_path: lib/sprout_engine/boundaries

strict_input: !Env { name: WANDERLAND_STRICT_INPUT, default: false }

grids:
  grocery_walk:
    max_ticks: 10
    entries:
      pet-store-fetch:
        bag: []
        list: ["cat food", "milk"]
    exits:
      home: {}
    cells:
      pet-store-fetch:
        boundary: oculus_fence
        args:
          fence_id: nita-buy
          params: { item: "cat food", quantity: 1, unit: "bag" }
        connections: { east: pet-store-tell }
      pet-store-tell:
        boundary: emit_signal
        args:
          action: oculus-refresh
          target: dag-board
          slug: dag-tutorial
          section: pet-store
        connections: { east: grocer-fetch }
      grocer-fetch:
        boundary: oculus_fence
        args:
          fence_id: nita-buy
          params: { item: "milk", quantity: 1, unit: "carton" }
        connections: { east: grocer-tell }
      grocer-tell:
        boundary: emit_signal
        args:
          action: oculus-refresh
          target: dag-board
          slug: dag-tutorial
          section: grocer
        connections: { east: home }
      home:
        boundary: emit_signal
        args:
          action: oculus-refresh
          target: dag-board
          slug: dag-tutorial
          section: home

routes:
  /echo:
    method: get
    boundary: echo
    name: echo

  /run:
    method: post
    name: run
    chain:
      - boundary: run_grid
        args:
          grid: !Reference grids.grocery_walk
      - boundary: collect_grid_state
        args:
          from: run_grid

  /run/step:
    method: post
    name: run-step
    chain:
      - boundary: run_grid
        args:
          grid: !Reference grids.grocery_walk
      - boundary: collect_grid_state
        args:
          from: run_grid

Anatomy:

The level reads pet-store-fetch → pet-store-tell → grocer-fetch → grocer-tell → home — five cells, five ticks for one item to walk to harvest.

2. Write the oculus_fence boundary

Before the code, a note on reading nested input. BoundaryInput's dot syntax wraps the top level — input.args reads the args field through method_missing, validates against the declared input_shape, and hands back the raw value. Once that value is a Hash, you're back in plain Ruby; input.args.fence_id is Hash#fence_id, which doesn't exist. The framework's path-based accessors live a level up:

The dot stops at one level; below that, the path-based accessor lets the validator's input_shape declarations and the call site agree on what's required without needing a wrapper layer that re-implements method_missing for every nested Hash.

A second namespacing note: signals fully qualify as Wanderland::Signal. Bare Signal resolves to Ruby's built-in Signal module (Unix signal handling — Signal.list, Signal.trap), which doesn't have ok / halt / emit. The framework's own boundaries always write Wanderland::Signal.ok(...); a site boundary nested in SproutEngine::Boundaries has nothing in scope that re-exports the wanderland-side constant.

Create lib/sprout_engine/boundaries/oculus_fence.rb:

# frozen_string_literal: true

require "net/http"
require "uri"
require "json"

module SproutEngine
  module Boundaries
    class OculusFence
      include Wanderland::Boundary

      IDENTITY = Wanderland::Identity.new(
        id: "boundary:oculus_fence",
        name: "OculusFence",
        roles: [:boundary],
        type: :service,
        scopes: [:grid]
      ).freeze

      boundary :oculus_fence,
        identity: IDENTITY,
        requirements: [],
        capabilities: [:fence_execution],
        description: "Execute an oculus fence; record the output in the item's bag",
        env: {
          OCULUS_BASE: "https://i.loss.dev",
          OCULUS_USER: nil,
          OCULUS_PASS: nil
        },
        input_shape: {
          args: {
            fence_id: true,
            params: {
              item: true,
              quantity: true,
              unit: true
            }
          }
        },
        output_shape: {
          fence_id: true,
          result: true,
          session_id: true,
          bag: true
        }

      def call(input)
        fence_id = input.find_or_fail("args.fence_id")
        params   = input.find_or_fail("args.params")
        env      = input["context"]["env"]

        uri = URI("#{env["OCULUS_BASE"]}/api/oculus/fences/#{fence_id}/execute")
        req = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
        req.basic_auth(env["OCULUS_USER"], env["OCULUS_PASS"]) if env["OCULUS_USER"]
        req.body = JSON.generate(params: params)

        res  = Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") { |h| h.request(req) }
        body = JSON.parse(res.body)

        bag = Array(input["context"]["bag"]).dup
        bag << {
          "item"     => params["item"],
          "quantity" => params["quantity"],
          "unit"     => params["unit"]
        }

        Wanderland::Signal.ok(
          fence_id:   fence_id,
          result:     body["data"],
          session_id: body.dig("metadata", "session_id"),
          bag:        bag
        )
      end
    end
  end
end

Anatomy:

3. Write the emit_signal boundary

Create lib/sprout_engine/boundaries/emit_signal.rb:

# frozen_string_literal: true

module SproutEngine
  module Boundaries
    class EmitSignal
      include Wanderland::Boundary

      IDENTITY = Wanderland::Identity.new(
        id: "boundary:emit_signal",
        name: "EmitSignal",
        roles: [:boundary],
        type: :service,
        scopes: [:grid]
      ).freeze

      boundary :emit_signal,
        identity: IDENTITY,
        requirements: [],
        capabilities: [:signal_emission],
        description: "Emit a :signals:web:<action> crossinng carrying the configured payload",
        input_shape: {
          args: {
            action: true,
            target: true
          }
        },
        output_shape: {
          "_type_addr": true,
          "signal_emitted": true,
          "payload": true
        }

      def call(input)
        args = input.args
        action = args.action
        payload = args.reject { |k, _| k.eql? 'action' }

        Signal.emit(
          ":signals:web:#{action}",
          signal_emitted: true,
          payload: payload
        )
      end
    end
  end
end

Anatomy:

4. Boot the engine

Reload from a fresh terminal:

bundle exec rake server

Two new boundaries land in the registry. rake boundaries confirms:

bundle exec rake boundaries | grep -E "oculus_fence|emit_signal"
# oculus_fence → SproutEngine::Boundaries::OculusFence
# emit_signal  → SproutEngine::Boundaries::EmitSignal

/inspect/route/run shows the route's compiled chain:

curl -s http://localhost:9295/inspect/route/run | python3 -m json.tool
# {
#   "name": "run",
#   "method": "post",
#   "path": "/run",
#   "user_chain": ["run_grid"],
#   "compiled_chain": ["enforce_denials", "run_grid", "verify_route", "trace_emit", "format", "seal"],
#   "scenarios": [],
#   ...
# }

user_chain: ["run_grid"] — the grid: shorthand normalizes to one slot. The framework's defaults wrap it the same way they wrap any chain.

5. Walk the grid — POST /run

In a second shell, cold-start the run:

curl -s -X POST http://localhost:9295/run \
     -H "Content-Type: application/json" \
     -d '{}' | python3 -m json.tool
# {
#   "value": {
#     "bag": [
#       { "item": "cat food", "quantity": 1, "unit": "bag" },
#       { "item": "milk",     "quantity": 1, "unit": "carton" }
#     ],
#     "list": ["cat food", "milk"]
#   },
#   "events": [
#     { "type_addr": ":signals:web:oculus-refresh",
#       "result": { "signal_emitted": true,
#                   "payload": { "target": "dag-board", "slug": "dag-tutorial", "section": "pet-store" } } },
#     { "type_addr": ":signals:web:oculus-refresh",
#       "result": { "signal_emitted": true,
#                   "payload": { "target": "dag-board", "slug": "dag-tutorial", "section": "grocer" } } },
#     { "type_addr": ":signals:web:oculus-refresh",
#       "result": { "signal_emitted": true,
#                   "payload": { "target": "dag-board", "slug": "dag-tutorial", "section": "home" } } }
#   ],
#   "items": []
# }

Five ticks: oculus_fence at pet-store-fetch (bag gets cat food), emit_signal at pet-store-tell (signal #1), oculus_fence at grocer-fetch (bag gets milk), emit_signal at grocer-tell (signal #2), emit_signal at home (signal #3, item harvested). items: [] means everything's been harvested; value is the projection across the item's sub-stream; events is the :signals:* slice — both produced by collect_grid_state reading the run_grid fork's cells.

6. Step the grid — POST /run/step

Cold-start with a single tick:

curl -s -X POST http://localhost:9295/run/step \
     -H "Content-Type: application/json" \
     -d '{}' | python3 -m json.tool
# {
#   "value": { "bag": [{ "item": "cat food", "quantity": 1, "unit": "bag" }], "list": [...] },
#   "events": [],
#   "items": [
#     {
#       "id": "item-7c1d…",
#       "position": "pet-store-tell",
#       "context": [
#         { "boundary": "spring",       "result": { "bag": [], "list": [...] }, "...": "..." },
#         { "boundary": "oculus_fence", "result": { "bag": [...] },             "...": "..." }
#       ]
#     }
#   ]
# }

The spring seeded the item with one crossing; one tick at pet-store-fetch fired oculus_fence and appended a second crossing carrying the cat-food purchase. The item advanced to pet-store-tell. events is empty for this tick because the next signal hasn't fired yet. items carries the in-flight item, including its full context — the source of truth for everything that's happened to it so far.

POST it back to step again:

curl -s -X POST http://localhost:9295/run/step \
     -H "Content-Type: application/json" \
     -d '{
       "items": [
         {
           "id": "item-7c1d…",
           "position": "pet-store-tell",
           "context": [
             { "boundary": "spring",       "result": { "bag": [], "list": ["cat food", "milk"] } },
             { "boundary": "oculus_fence", "result": { "bag": [{"item":"cat food","quantity":1,"unit":"bag"}] } }
           ]
         }
       ]
     }' | python3 -m json.tool
# {
#   "value": { "bag": [...], "list": [...] },
#   "events": [
#     { "type_addr": "signals:web:oculus-refresh",
#       "payload": { "target": "dag-board", "section": "pet-store", "...": "..." } }
#   ],
#   "items": [{ "id": "item-7c1d…", "position": "grocer-fetch", "context": [/* 3 crossings now */] }]
# }

The pet-store signal fires this tick; the item moves to grocer-fetch with its context now three crossings deep. Repeat three more times — events per call carries just that tick's signals, the item's context grows by one crossing per tick — until items: [] lands. Five steps to harvest.

This is exactly what the puzzle UI does: keep reposting items from the previous response, fan events[] out via lanternBus after each call, render value as the projected state. The context on the wire is opaque to the UI; treat it as a token to round-trip.

7. Pause and resume — input prompting

To see the pause mechanic, drop the quantity field from grocer-fetch.args.params:

grocer-fetch:
  boundary: oculus_fence
  args:
    fence_id: nita-buy
    params: { item: "milk", unit: "carton" }   # quantity removed
  connections: { east: grocer-tell }

Restart and POST /run again:

curl -s -X POST http://localhost:9295/run -H "Content-Type: application/json" -d '{}' | python3 -m json.tool
# {
#   "value": {
#     "bag": [{ "item": "cat food", "quantity": 1, "unit": "bag" }],
#     "list": [...]
#   },
#   "events": [
#     { "type_addr": "signals:web:oculus-refresh", "payload": { "section": "pet-store", "...": "..." } }
#   ],
#   "items": [
#     {
#       "id": "item-7c1d…",
#       "position": "grocer-fetch",
#       "context": [/* entry seed, pet-store-fetch, pet-store-tell */],
#       "awaiting": {
#         "boundary": "oculus_fence",
#         "missing": ["quantity"],
#         "schema": { "quantity": true }
#       }
#     }
#   ]
# }

Two ticks ran cleanly (pet-store-fetch and pet-store-tell each appended a crossing). On the third tick, the item paused at grocer-fetch because oculus_fence declares input_shape.args.params.quantity required. awaiting names the boundary, the missing keys, and the slice of input_shape the UI needs to render a form against. The item's context is intact — every crossing up to the pause point is still on the item, ready to ride the next request back.

The UI renders something like [item-7c1d…] · grocer-fetch · oculus_fence · quantity: [____]. On submit, it reposts /run/step with the same item and args_override populated:

curl -s -X POST http://localhost:9295/run/step \
     -H "Content-Type: application/json" \
     -d '{
       "items": [
         {
           "id": "item-7c1d…",
           "position": "grocer-fetch",
           "context": [/* prior crossings, unchanged */],
           "args_override": { "params": { "quantity": 1 } }
         }
       ]
     }' | python3 -m json.tool

run_grid deep-merges args_override into the cell's args before firing the boundary; the input_shape is satisfied; the fence call lands; the resulting crossing appends to the item's context; the item advances. Repeat steps until harvest.

Restore quantity in the YAML before moving on so the rest of the tutorial walks cleanly.

8. Add a scenario

Append a scenarios: block to config.yml:

scenarios:
  run:
    full_walk:
      description: "Cold start completes the grocery walk; bag carries cat food and milk; three scene swaps emit."
      input:
        params: {}
      expected:
        items: []
        value:
          bag:
            count: 2
            in_order:
              - { item: "cat food", quantity: 1, unit: "bag" }
              - { item: "milk",     quantity: 1, unit: "carton" }
          list: ["cat food", "milk"]
        events:
          count: 3
          in_order:
            - { type_addr: "signals:web:oculus-refresh", payload: { section: "pet-store" } }
            - { type_addr: "signals:web:oculus-refresh", payload: { section: "grocer" } }
            - { type_addr: "signals:web:oculus-refresh", payload: { section: "home" } }

Anatomy:

Restart the engine and confirm /inspect/scenarios registers it:

curl -s http://localhost:9295/inspect/scenarios | python3 -m json.tool
# {
#   "scenarios": {
#     "run": {
#       "full_walk": {
#         "description": "Cold start completes the grocery walk; ...",
#         "has_input": true,
#         "has_expected": true,
#         "input": { "params": {} },
#         "expected": { ... }
#       }
#     }
#   }
# }

9. Verify with --type test

Stop puma and run the test runner against the same config:

bundle exec wanderland --type test config.yml
# config scenarios
#   run
#     ✓ full_walk
# 1 scenario, 0 failures

--type test walks every scenario discovered by Wanderland::Scenario.from_config(runtime). For each, it dispatches the route with X-Wanderland-Verify: <scenario-name> set; verify_route (the framework injection) compares the route's response against expected: via ShapeMatcher. The exit code is 0 on a clean pass, non-zero if anything failed.

Filter to one scenario with --scenario full_walk or one route with --route run. Pin the same scenarios in your rspec suite by adding to spec/scenarios_spec.rb:

require "spec_helper"
runtime = Wanderland.boot(File.join(__dir__, "..", "config.yml"))
Wanderland::Scenario.rspec_describe!(runtime)

bundle exec rake spec walks the same cases under rspec, with failure output coming straight from ShapeMatcher.

What's next

Site Audit

wanderland.dev

oculus-view: fence: fence execute HTTP 404

3 Write The Emit Signal Boundary