wisp is the compute-plane router — the scheduler tier that sits under the echo/img launchers. It classifies a work request against a registered job type, sizes it from foxfire's usage statistics, reserves Kueue quota, and launches the Nextflow driver as a Kubernetes Job gated by its own queue. It runs on Kubernetes (CoreWeave dev, OrbStack local) as a namespace-internal service.
POST /submit — classify a work request (job_type + immediacy + opaque params), size each process, pick the worker queue, and launch the driver Job. Returns the run id, queue, size, and per-process plan.POST /preview — the same classification with no launch: the routing decision and the statistics that drove it, for a dry run.GET /queues — the Kueue topology (cohort → ClusterQueue → LocalQueue) and the workloads wisp is routing, curated for the dashboard.GET /config — the template library and sizing tables the router reads.GET /plan/:run_id, GET /placement/:run_id, GET /runs/:run_id/reconcile — the plan wisp emitted, where the reaper saw the run land, and the drift between them.POST /admin/clear-failed — sweep Failed task pods (their record lives on in foxfire).A launcher turns a UI action into work, but where that work runs — what size, which queue, on-demand or spot, gated by how much quota — is a scheduling decision that needs the fleet's history and the cluster's quota model. wisp is that decision. It reads foxfire's per-process p95 statistics to size a run, maps the caller's immediacy to a worker queue family, and hands the driver Job to Kueue with the resources it reserved. The template library is the whole registration surface: a job type is one entry in config/templates.yml, no router code.
Wanderland::Engine. The submit path is a chain: submit (validate, mint the run id) → foxfire_envelope (fetch the job type's rolled-up statistics) → profile (size each process to a tier, pick the queue) → classify (render the driver Job manifest, launch it, emit the plan to foxfire).spot: safe|unsafe outright; immediate work never spots.immediate | deferred | batch) crossed with the compute family (cpu/gpu) selects the worker queue — workers-{immediate,deferred,batch}-{cpu,gpu}. The driver Job itself is gated in a separate drivers queue (cohort drivers-cq), so drivers and their task pods are admitted independently.classify renders a batch/v1 Job in vivarta-drivers (the advaita/<driver> image, nextflow-runner SA, a per-run overlay ConfigMap merged over the workflow's config) plus the mounts the template declares — the shared nextflow-work RWX volume, the always-shared nextflow-cache volume at /nxf-cache (NXF_CACHE_DIR), and any workflow data volumes. Task pods land in vivarta-workers.resume_of: <run_id>; classify resolves that run's Nextflow session from foxfire's lineage record and appends -resume <session-id>, so a resubmission continues the prior session's cache. Unresolvable lineage is a 422, never a silent fresh run.classify posts the plan (per-process queue, requests, the statistics it sized from) to foxfire. The reaper records where each pod actually landed; foxfire joins plan + placement by run id for the drift view.kubectl (apply the driver Job, read the Kueue objects) and http (reach foxfire).advaita/wisp (ECR 789905347053.dkr.ecr.us-west-2.amazonaws.com/advaita/wisp), the engine-service image (ruby:3.4-slim + the kubectl binary; clones wanderland-core + the console/lantern pack as siblings; boots wanderland --type http config.yml). Built and pushed by wisp/vivarta.yml.http), probes /health, serviceAccountName: wisp./queues, /preview, /submit proxies) and by puck.vivarta-drivers.| Env | Context | Image stage |
|---|---|---|
| local | orbstack |
build → ECR (arm64) |
| dev | useast13-dev |
build → ECR (amd64) |
Two one-shot drones, run before the first deploy — not part of the image pipeline:
wisp/drones/bootstrap.yml — installs Kueue, applies the compute-plane substrate (namespaces, resource flavors, cluster/local queues, wisp RBAC, the cluster-scoped queue-reader grant), and the storage: the shared nextflow-work volume and the always-shared nextflow-cache volume, per substrate (OrbStack hostPath pairs; CoreWeave dynamic shared-vast PVCs). On non-OrbStack contexts it also applies the ESO-managed ecr-creds pull secrets.wisp/drones/workers-bootstrap.yml — the nextflow-runner SA + cross-namespace RBAC, the VAST static-clone that unifies nextflow-work (and nextflow-cache, and any named workflow data PVC) across vivarta-drivers/vivarta-workers/vivarta-echo, foxfire's signing key copied into the driver namespace, and fresh ecr-creds in the split namespaces.config/templates.yml is where every job type is declared: the driver image, the workflow + config paths, the ERB-rendered run options, the volume mounts, and the resource class (cold-start defaults, min/max clamps, spot posture, gpu declaration, the process list). Adding a job type is one entry; the router learns everything it knows about a workload from it. The registered set spans the synth setlist (the karaoke stress workloads), img.recontrast_run (the metabolomics pipeline), and echo.fibfizz (the toy).
config.yml: the boundary chain, the kubectl + http adapters, and route args via !Env — the driver namespace/SA, the foxfire URL, the signing-key secret, the always-shared cache PVC name.config/ as their own reviewable files, read as input["config"].*.wisp/vivarta.yml (containerized_deployment; docker_build → ECR; k8s_provisioner; ClusterIP only). Platform forced per env — local arm64, dev amd64.| Dependency | Use | Endpoint / id |
|---|---|---|
| foxfire | per-process sizing statistics (envelope); receives the plan | foxfire.vivarta-echo.svc.cluster.local:9311 |
| Kueue | admission + quota for the driver and task queues | ClusterQueues / LocalQueues |
| ECR | image registry | advaita/wisp |
| Kubernetes API | apply driver Jobs, read Kueue objects | via wisp SA |
| driver images | launched as one-shot Jobs | advaita/synth-driver, advaita/ortho-driver, advaita/nf-driver |
| Dependent | Use |
|---|---|
| echo-backend | proxies /queues, /preview, and /submit (the launcher's wisp-routed path) |
| puck | launches drone sorties that submit waves of work to /submit |
| reaper | records placement for the runs wisp launches; foxfire joins plan + placement |
New work cannot be routed or launched through the compute plane — submits fail and the queue view goes stale. Driver Jobs already admitted continue; the reaper and foxfire keep recording them. Blast radius is new launches and the routing view, not in-flight compute.
wisp (git@github). Ruby / wanderland-core engine; boundaries under lib/wisp/boundaries/; the sizing math in lib/wisp/; the Kueue substrate + RBAC under k8s/; the lab sorties (the karaoke scenario library) under drones/lab/.