PKB on CoreWeave via the Compute Plane

ADV-455 · case task-0e952514-c8e0-4a13-8a52-b85fa8550ecf · status: design, nothing deployed

Deploy the Protein Knowledge Base (PKB) Docker workflow on CoreWeave by running it as a driver on the existing compute plane — Nextflow schedules, advaita-service-wisp routes, Kueue queues, foxfire records — rather than hand-orchestrating Jenkins jobs. Full platform model: advaita-nextflow-router-data-model.

Summary

PKB's acquire → governed-build → promote pipeline becomes a Nextflow workflow registered as a wisp job_type. Each phase is a task pod; the pod carries a securityContext (the container contract), volume mounts, and an egress-class label that a standing NetworkPolicy selects. The build steps that are traditional CI/CD (building the PKB image) shell out to Wanderland, which already emits signed traces. Both trace streams — Nextflow's (via nf-foxfire) and Wanderland's — land in foxfire.

We prove the whole shape as a mock in vivarta-examples first (that is where example projects live), on the Foxfire+Wanderland base image, then fill in Grant's real PKB container when the repo lands.

Architecture map

Layer	Role	Repo
Nextflow driver	one-shot k8s Job per run; expands the workflow into task pods	driver image (nf-driver-base + workflows)
wisp	POST /submit → classify → size from history → launch driver gated by Kueue. Registration = one config/templates.yml entry, no code	wisp
Kueue	driver Job → drivers cohort; task pods → workers-<immediacy>-<compute>	wisp/k8s/kueue
foxfire	signed (EC-P256) plan / placement / trace ledgers joined on run_id	foxfire
synth	reference driver project to copy	synth

How PKB maps

• Two job_types: pkb.build (ACQUIRE + BUILD → immutable candidate; immediacy: batch; BUILD process spot: unsafe) and pkb.promote (separate, human-gated submit after candidate review).
• Namespace: vivarta-workers for task pods (decided 2026-07-17, movable later). PVCs, the KEGG secret, and the NetworkPolicies live there.
• Container contract lives in the workflow's baked nextflow.config as native Nextflow pod directives per process: securityContext (runAsNonRoot, UID/GID 10001, readOnlyRootFilesystem, no privilege escalation), volumeClaim mounts (/inputs ro, /var/lib/pkb rw, /scratch/pkb rw), process.container = digest-pinned PKB image.
• Egress posture = a per-process pod label. ACQUIRE → egress: controlled; BUILD → egress: deny. Standing policies select the label: NetworkPolicy default-deny+DNS (= BUILD) and CiliumNetworkPolicy toFQDNs (= ACQUIRE, approved sources + Nexus + object store). Both templates cribbed from pkb-v001's k8s/base/network-policy.yaml and cilium-egress.yaml.
• Approved sources ride the submit data: map (kegg: ftp://…); Nextflow scheme-dispatches; the Cilium policy is what actually gates reachability.
• The overlay stays available but unused for this: the data model confirms nextflow -c base -c /etc/wisp/overlay.config merges any Nextflow scope, so per-run variation is possible later. The static contract belongs in the baked config, so wisp/foxfire/Kueue need no change.

The trace question — Wanderland Crossing vs foxfire envelope

Goal: send Wanderland's per-boundary traces to foxfire so CI/CD steps done in Wanderland are captured next to the Nextflow trace. The two are the same envelope family at two hardening levels — not byte-identical.

Wanderland Crossing (wanderland-core/lib/wanderland/crossing.rb) — one per boundary execution: boundary, from_addr, to_addr, type_addr, trace, result, requirements, capabilities, at, duration_ms, signature. Signs canonical (the whole record minus signature/duration, sorted-key JSON) under from_addr via Activities::PKI (EC P-256). trace = the merkle causal link (previous crossing's signature). Sink today = per-run SQLite .db files under a trace root (boundaries/trace/trace_stores.rb), read by trace_query/trace_runs/trace_replay.

foxfire envelope (foxfire/docs/event-envelope.md, boundaries/ingest_signed.rb): key_id, to, from, type, payload(JSON string), trace_hash, signature. Signs signable = key_id\nto\nfrom\ntype\ntrace_hash — addresses + a content hash of the payload, not the payload bytes. foxfire's ingest boundary literally include Wanderland::Boundary, and its signing "matches Wanderland::Activities::PKI."

Same: four-address model, type-classified, signed, one EC-P256 PKI, and foxfire's receiver IS a Wanderland boundary. The instinct that "foxfire input is the same schema as the engine" is right at the envelope level.

Different, and why:

• The fourth field collides in name only — Wanderland trace = merkle link between crossings (causal chain); foxfire trace_hash = SHA-256 of the payload (tamper-evidence). Different jobs.
• Signing scope — Wanderland signs the whole canonical record (Ruby-to-Ruby, it controls canonicalization both ends); foxfire signs addresses + payload-hash as a newline string so a JVM emitter and a Ruby verifier agree on the signed bytes without a shared JSON canonicalizer.
• key_id indirection (rotation-stable handle) vs Wanderland signing under from_addr directly.
• Payload inner contract — foxfire folds runId/runName/event/utcTime/trace/metadata (Nextflow lifecycle); a Wanderland crossing's result is an arbitrary boundary Signal.

foxfire's envelope is the cross-runtime, cross-process hardening of the same crossing. The engine's crossing never leaves the process, so it signs the canonical record and chains by signature; foxfire's must survive a JVM→Ruby hop and key rotation, so it adds key_id + payload-hash + string-payload.

The hole to fill (two ends):

• wanderland-core: an external trace sink, env-gated (e.g. WANDERLAND_TRACE_SINK=<foxfire>/events/signed + a key_id), that per crossing builds the foxfire envelope form (map from_addr/to_addr/type_addr → to/from/type; serialize result → payload string; trace_hash = SHA256(payload); sign the newline signable via the same PKI) and POSTs it — alongside the existing SQLite store.
• foxfire: accept a boundary-lifecycle event type with a boundary-shaped payload (or the sink maps boundary executions onto existing event types and supplies runId = the Wanderland request/trace id, event = the boundary name).

The mock (in vivarta-examples)

Prove the shape end-to-end before Grant's container exists:

• A Nextflow workflow with ACQUIRE + BUILD processes that call a stand-in pkb-container (a shell that writes a fake candidate + completion-manifest.json), on the Foxfire+Wanderland base image (built by others in vivarta-platform — cribbed when ready; Ruby-on-Java layering is a fill-in).
• BUILD's image-build step shells to a Wanderland flow (reuse docker_build/publish_image pipeline primitives), whose signed traces flow to foxfire via the sink.
• A pkb.build template in wisp, task pods in vivarta-workers, the two NetworkPolicies, and stub PVCs.
• Verify: foxfire holds one run_id with BOTH the Nextflow task trace AND the Wanderland boundary trace; the egress labels land on the pods; the securityContext is honored.

Steps

• Repo access for advaita-corporation/protein-knowledge-base (Grant) — unblocks exact CLI + build-spec schema. Mock proceeds without it.
• Scaffold the example project in vivarta-examples (crib synth): driver/, workflows/pkb/{main.nf,nextflow.config}, registration/{templates.snippet.yml,envelopes/}.
• Write workflows/pkb/nextflow.config — securityContext + volumeClaim mounts + per-process egress label; stand-in pkb-container script for ACQUIRE/BUILD.
• Standing k8s in vivarta-workers: shared-vast RWX PVC for /var/lib/pkb, scratch, KEGG secret stub, the default-deny NetworkPolicy + toFQDNs CiliumNetworkPolicy (crib pkb-v001).
• Wanderland image-build flow (shell to docker_build/publish_image), invoked from the BUILD process.
• wanderland-core trace sink → foxfire (env-gated) + foxfire boundary-event acceptance. Prove one run_id carries both traces.
• Register pkb.build (+ later pkb.promote) template; POST /submit; read the fold in foxfire.
• Deploy ladder: OrbStack → k3s purple-plum-playhouse (real spot/on-demand node-class) → CoreWeave.
• Swap the stand-in for Grant's real PKB container; pin resource shape to the v20 build spec; wire pkb.promote gate.

Blocked / standing

• Blocked: protein-knowledge-base repo (404 to gfawcett-advaitabio) — exact pkb-container CLI + pkb_v20_0_cutoff_2026_07_07_build_spec.json schema. Grant notified.
• Standing: readOnlyRootFilesystem vs Nextflow .command.* staging (expected OK — writes go to workDir PVC + /tmp — must test); /var/lib/pkb size + retention on shared-vast; /scratch emptyDir vs PVC; foxfire payload contract for boundary events; ECR region pull-secret (us-west-2 vs us-east-1); promotion approval mechanism; whether some licensed inputs mount from existing read-only PVCs vs download.
• Design sits at: architecture decided and grounded in code; trace path understood with a named two-end hole; mock scaffolding is the next build. Someone else builds the Foxfire+Wanderland base image in vivarta-platform.

Wiring the emitter — an injection, not top-level machinery

Graeme's call, and it's right: don't add a bespoke trace sink to Core. Build the Wanderland→foxfire step as a boundary and compile it into the chain via the injection registry, declared in the site vivarta.yml. It fires every slot, the way enforce_denials does.

Doc-vs-code note. wanderland-core-interceptors (Draft, 2026-04-09) describes this as an "interceptor" with before/after hooks on Boundary.execute + run_levels. The code implements it as injections, not interceptors — wanderland-core/lib/wanderland/injection.rb (InjectionRegistry) + the register_injections boot boundary, applied at route-compile; no Interceptor module exists. Use the injection mechanism (what the primer documents and what enforce_denials / trace_emit already use). Position vocabulary: first / last / interleave / {before: name} / {after: name}.

The emitter:

• Boundary foxfire_emit — reads the crossing from context, transforms Wanderland shape → foxfire envelope (from_addr/to_addr/type_addr → from/to/type; result → payload JSON string; trace_hash = SHA256(payload); sign the newline signable via Activities::PKI), POSTs via the :http adapter to <foxfire>/events/signed. A thin wrapper around http + PKI — the whole content is one transform + one emission.
• Declared in vivarta.yml injections: as { boundary: foxfire_emit, position: interleave } (fires between every slot, like the denial chain) — or { boundary: foxfire_emit, after: trace_emit } to sit right after the core trace injection.
• Swappable shapes: a different emitter (different transform/endpoint) is a different injection entry. Pick one, or several.

trace_emit already exists as a core injection (position last) recording the crossing; foxfire_emit is its externalizing sibling. No engine change beyond adding the boundary + the config line. The env gate is just "is the injection present + the :http adapter's foxfire URL set." Open sub-item: extend trace_emit to take an endpoint vs add foxfire_emit beside it — the latter keeps Core untouched (preferred). This supersedes the earlier "external trace sink in wanderland-core" framing in the trace-question section above.