Wanderland

Vivarta Jenkins Tutorial: Secrets via a Tools Image

This is the second tutorial in the vivarta-on-k8s series. The first one put a Jenkins controller on a local Kubernetes cluster and got an ephemeral build pod running. This one teaches the secrets half of the same control surface, and the pattern for multiple pod templates: we'll bake a small tools image, register it as a second pod template, and point a project at it through the agent_label field in projects.yml.

The image we'll build is called my-cool-tools — three files inside vivarta-jenkins/images/cool-tools/: a Dockerfile, a config.yml, and one boundary .rb. wanderland-core gets gem-installed globally inside the image, so the wanderland binary is on PATH and our pipeline body's wanderland --type cli config.yml <route> runs without any bundle-exec dance. No Gemfile in the project. The secret is an HTTP basic-auth pair; Jenkins binds it as two env vars around the run step; the config's !Environment resolvers thread them through.

When you're done you'll be able to:

About thirty minutes the first time, ten a second.

Where this fits

Three ways to give a Jenkins build access to a tool, and it's worth naming each before we pick.

Grow the agent image. Every project's deploy runs on the same vivarta-agent image, and every utility lives there too. Cleanest when the toolchain is small and stable. Gets uncomfortable as the agent grows — curl, jq, aws, kubectl, gcloud each slow every project's pull, and a single bug in the agent halts everyone.

Ship the tool with the project. Every project repo carries its own Dockerfile or scripts; the agent stays generic. Cleanly factored, but every utility one-off becomes a per-repo bake-and-publish cycle.

Bake a tools image as a second pod template. What we're doing. A focused image, registered alongside the agent in JCasC, selected by a project through one field — agent_label: cool-tools. Adding a third or fourth tools image is the same recipe. The agent stays focused on project deploys; specialised work runs on specialised pods.

What we're going to build

vivarta-jenkins/images/cool-tools/
  Dockerfile          # ruby:slim + wanderland-core installed globally
  config.yml          # one route + the http adapter mount
  lib/cool_tools/boundaries/read_node.rb

That's the whole image source. No Gemfile, no Rakefile, no bin/ wrapper, no build script. The wanderland binary is on PATH because gem install wanderland-core puts it there, and the HTTP adapter ships in the same gem.

The image gets built by adding one entry to vivarta-jenkins's own vivarta.yml pipeline. A pod template, a deployment env-var, a project entry, a k8s secret round out the wiring — most of them one block of yaml each.

What the secrets binding gives us

The control surface lives in vivarta-jenkins/jcasc/projects.yml. A project's entry can carry an optional secrets: list. Each entry asks the pipeline to bind one Jenkins credential for the duration of the dry-run + run steps:

secrets:
  - id: oculus-creds-${WANDERLAND_ENVIRONMENT}
    bind: usernamePassword
    env:
      username: OCULUS_USER
      password: OCULUS_PASS

Three things happen:

Supported bind types today: file (env var holds a path to a temp file with the secret's content), string (env var holds the secret value directly), usernamePassword (two env vars). All three map to the Jenkins withCredentials binding of the same name.

The ${WANDERLAND_ENVIRONMENT} substitution

${WANDERLAND_ENVIRONMENT} inside a secrets[].id is resolved at build time from the chosen environment. One secrets: entry covers every environment — the k8s secret naming convention does the per-env work: oculus-creds-dev, oculus-creds-staging, oculus-creds-prod.

Step 0. Prerequisites

Step 1. Clone the repo

git clone git@github
cd vivarta-jenkins

If you already have a working checkout from the parent tutorial, git pull it.

Step 2. Create the image directory

mkdir -p images/cool-tools/lib/cool_tools/boundaries

Three files go in there, top to bottom.

Step 3. The boundary

# images/cool_tools/lib/cool_tools/boundaries/read_node.rb
require "base64"

module CoolTools
  module Boundaries
    class ReadNode
      include Wanderland::Boundary

      IDENTITY = Wanderland::Identity.new(
        id:     "boundary:read_node",
        name:   "ReadNode",
        roles:  [:boundary],
        type:   :service,
        scopes: %i[read network]
      ).freeze

      boundary :read_node,
        identity:     IDENTITY,
        requirements: [:read],
        capabilities: [:read],
        description:  "Read an Oculus node via basic auth over HTTP",
        input_shape: {
          "args" => {
            "base_url" => { required: true },
            "user"     => { required: true },
            "pass"     => { required: true }
          },
          "params" => {
            "slug" => { required: true }
          }
        },
        output_shape: { "status" => true, "body" => true },
        adapters: [:http]

      def call(input)
        base_url = input.find_or_fail("args.base_url")
        user     = input.find_or_fail("args.user")
        pass     = input.find_or_fail("args.pass")
        slug     = input.find_or_fail("params.slug")

        url  = "#{base_url}/api/oculus/node/#{slug}"
        auth = "Basic " + Base64.strict_encode64("#{user}:#{pass}")

        response = input.adapter(:http).get(url, headers: { "Authorization" => auth })

        case response["status"]
        when 200 then Wanderland::Signal.ok(status: 200, body: response["body"])
        else          Wanderland::Signal.halt(status: response["status"], error: response["body"])
        end
      end
    end
  end
end

Anatomy:

The boundary doesn't shell out, doesn't construct argv, and doesn't interpolate user input into a command string. The HTTP shape is the contract, and the adapter owns every byte going on or off the wire.

Step 4. The config

# images/cool-tools/config.yml
service: cool-tools
boundary_path: lib/cool_tools/boundaries

adapters:
  http:
    kind: http
    mode: !Env { name: WANDERLAND_HTTP_MODE, default: live }

routes:
  /read-node/:slug:
    method: get
    boundary: read_node
    name: read-node
    args:
      base_url: !Env OCULUS_BASE_URL
      user:     !Env OCULUS_USER
      pass:     !Env OCULUS_PASS

Anatomy:

The boundary never sees Jenkins, k8s secrets, or ${WANDERLAND_ENVIRONMENT}. It sees two doorways: args (site-supplied — base_url, user, pass) and params (caller-supplied — slug). The HTTP adapter is mounted under :http and pulled at call time.

Step 5. The Dockerfile

# images/cool-tools/Dockerfile
FROM ruby:3.4-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
      build-essential ca-certificates git \
    && rm -rf /var/lib/apt/lists/*

# wanderland-core arrives via --build-context, gets built into a .gem
# and installed globally so the `wanderland` binary lands on PATH.
COPY --from=wanderland-core . /tmp/wanderland-core/
RUN cd /tmp/wanderland-core \
    && gem build wanderland-core.gemspec \
    && gem install --local wanderland-core-*.gem \
    && rm -rf /tmp/wanderland-core

WORKDIR /opt/cool-tools
COPY config.yml /opt/cool-tools/
COPY lib        /opt/cool-tools/lib/

WORKDIR /workspace

Anatomy:

curl is not in the image. The boundary talks to Oculus through Net::HTTP inside the HTTP adapter; no shellout, no command-line tool dependency to keep current.

Step 6. The build script

The image gets built by adding one entry to vivarta.yml's pipeline. The same containerized_deployment archetype that builds the controller and agent images iterates this one too — same docker_build → publish_image chain, different context path.

Open vivarta.yml. Under pipeline.artifacts.images, add a sibling to vivarta-jenkins-controller and vivarta-jenkins-agent:

pipeline:
  artifacts:
    images:
      # ...existing controller + agent entries...

      cool-tools:
        type: docker_image
        args:
          context_path: images/cool-tools
          image_name:   !Environment cool_tools_image_name
          image_tag:    !Reference pipeline.metadata.tags.version
          docker_build_args:
            - --build-context
            - wanderland-core=../wanderland-core

And add the image name to each environment block:

environments:
  local:
    # ...existing entries...
    cool_tools_image_name: host.docker.internal:5001/my-cool-tools

For incremental work on the image (rebuild + push without walking the whole pipeline), add a narrow route in vivarta.yml's routes: block:

routes:
  # ...existing routes...
  /build-cool-tools:
    method: post
    name: build-cool-tools
    chain:
      - boundary: docker_build
        args:
          source_path: images/cool-tools
          image_name:  !Environment cool_tools_image_name
          image_tag:   !Reference pipeline.metadata.tags.version
          docker_build_args:
            - --build-context
            - wanderland-core=../wanderland-core
      - boundary: publish_image
        args:
          image_name: !Environment cool_tools_image_name
          image_tag:  !Reference pipeline.metadata.tags.version

And the matching rake task in the Rakefile:

namespace "cool-tools" do
  desc "Build and push the cool-tools image"
  task :build do
    sh "#{W} build-cool-tools"
  end
end

Anatomy:

Walk it in dry-run before committing:

bundle exec wanderland --type cli vivarta.yml build-cool-tools --wanderland-dry-run

Two docker shell calls planned — build then push. Drop --wanderland-dry-run once the args look right.

Step 7. The pod template

Add a second templates[] entry under clouds.kubernetes.templates in jcasc/jenkins.yaml, sibling to the vivarta-agent block:

  - name: "cool-tools"
    namespace: "vivarta-ci"
    label: "cool-tools"
    serviceAccount: "jenkins-agent"
    yaml: |
      apiVersion: v1
      kind: Pod
      spec:
        containers:
          - name: vivarta
            image: ${COOL_TOOLS_IMAGE_REF}
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["999999"]
            tty: true
            resources:
              requests: { cpu: "200m", memory: "256Mi" }
              limits:   { cpu: "1",    memory: "1Gi" }

And add the env var to k8s/base/deployment.yaml's env: block on the controller container, with a default that matches the OrbStack path:

- name: COOL_TOOLS_IMAGE_REF
  value: host.docker.internal:5001/my-cool-tools:0.1.0

Anatomy:

Step 8. The project entry

Add to jcasc/projects.yml:

projects:
  - name: read_oculus_node
    description: Fetch an Oculus node — utility job, demonstrates the secrets shape.
    git_url: git@github
    config_path: images/cool-tools/config.yml
    agent_label: cool-tools
    route: read-node
    args:
      slug: why
    dry_run: false
    environments:
      mode: single
      from: [dev]
    secrets:
      - id: oculus-creds-${WANDERLAND_ENVIRONMENT}
        bind: usernamePassword
        env:
          username: OCULUS_USER
          password: OCULUS_PASS

Anatomy:

Step 9. The k8s secret

# oculus-creds-dev.yaml — gitignored, applied directly
apiVersion: v1
kind: Secret
metadata:
  name: oculus-creds-dev
  namespace: vivarta-ci
  labels:
    jenkins.io/credentials-type: usernamePassword
  annotations:
    jenkins.io/credentials-description: "Oculus creds for dev"
type: Opaque
stringData:
  username: wonderfullyweird
  password: thepurpleplumplayhouse

kubectl apply -f oculus-creds-dev.yaml

The kubernetes-credentials-provider plugin notices the labeled secret and surfaces it as a Jenkins credential with ID oculus-creds-dev immediately — no restart required.

Step 10. Apply, build, verify

Three commands — build the image, apply the JCasC and deployment changes, watch the controller roll:

Record and replay

The HTTP adapter the boundary pulls (input.adapter(:http)) carries a four-mode dispatch: live (real Net::HTTP), replay (recorded responses from disk), mock (scenario-stubbed responses), dry_run (logged, never invoked). Recording a live call once gives you a cassette the next replay-mode run serves offline — useful for CI, useful for demoing without an Oculus instance to hand, useful for catching regressions when the upstream's response shape drifts.

Three steps: record, inspect, replay.

Record. From the cool-tools image directory (vivarta-jenkins/images/cool-tools/), run the route once with --wanderland-record-to:

OCULUS_BASE_URL=https://i.loss.dev \
OCULUS_USER=wonderfullyweird \
OCULUS_PASS=downinfragglerock \
bundle exec wanderland --type cli config.yml read-node \
  --slug why \
  --wanderland-record-to spec/fixtures/cassettes

The call goes out as normal — live mode. The flag tells the adapter to write the response envelope as a YAML cassette on its way back. One file per unique (method, url, body, headers) interaction, named by SHA-1 of the canonical request shape.

Inspect. Look at what landed:

ls spec/fixtures/cassettes/http/
# 9e5c8879cb8265eae3b71928a68c2f4b7f0497dd.yml

cat spec/fixtures/cassettes/http/9e5c8879cb8265eae3b71928a68c2f4b7f0497dd.yml

---
request:
  method: GET
  url: https://i.loss.dev/api/oculus/node/why
  body:
  headers:
    Authorization: Basic d29uZGVyZnVsbHl3ZWlyZDpkb3duaW5mcmFnZ2xlcm9jaw==
response:
  status: 200
  body: |-
    {
      "slug": "why",
      "content": "..."
    }
  headers:
    content-type: application/json; charset=utf-8

YAML's literal-block scalar (body: |-) keeps the response body readable as itself — hand-edit the cassette when the upstream drifts and you'd rather follow than re-record. The basic-auth header is baked into the request key, so a cassette recorded with one credential pair won't replay against a different one (which is correct — that's a different interaction).

Replay. Flip mode at invocation time:

WANDERLAND_HTTP_MODE=replay \
OCULUS_BASE_URL=https://i.loss.dev \
OCULUS_USER=wonderfullyweird \
OCULUS_PASS=downinfragglerock \
bundle exec wanderland --type cli config.yml read-node --slug why

Same input, no network. The adapter computes the cassette key from the request, reads the YAML, returns the recorded response. A request the cassettes don't cover raises CassetteMissError with the file path the adapter looked for — drift fails loudly instead of silently hitting the wire.

!Env { name: WANDERLAND_HTTP_MODE, default: live } (Step 4) is the resolver wired into the adapter's mode:. The env var flips the same config at invocation; no edit, no rebuild.

Re-recording one interaction. Delete its cassette and re-run with --wanderland-record-to while back on mode: live. The one missing interaction lands; the rest stay frozen. A git diff against the cassette directory tells you exactly what changed in the upstream's response.

Mock for scenario-specific shapes. Replay covers "what does this boundary do against the real upstream." For "what does it do when the upstream returns a shape you can't reliably get on demand" — a 503, a malformed body, a node that doesn't exist — declare a scenario in config.yml:

scenarios:
  read-node:
    upstream_unavailable:
      description: "Oculus 503 surfaces as the upstream's status verbatim"
      input:
        params: { slug: why }
      expected:
        status: 503
      mocks:
        http:
          - request: { url: https://i.loss.dev/api/oculus/node/why }
            response: { status: 503, body: "service unavailable", headers: {} }

Run with WANDERLAND_HTTP_MODE=mock bundle exec wanderland --type test config.yml --scenario upstream_unavailable and the boundary sees the canned 503 instead of a real upstream. Scenario mocks are registered at dispatch top and cleared in an ensure block, so each scenario starts fresh.

See Adapter Modes for the deeper walk — cassette digest scheme, !Fixture to lift cassettes as mocks, !Merge to twist one field of a recorded response, the #client escape hatch when the uniform-hash facade doesn't expose what a boundary needs.

rake cool-tools:build   # docker_build + publish_image for cool-tools
rake jcasc:apply        # bake configmap + apply overlay + rollout restart

rake cool-tools:build walks the narrow /build-cool-tools route added in Step 6 — two docker shell calls (build, push) against the entries that live in vivarta.yml. rake jcasc:apply walks /apply-jcasc — bake_jcasc_configmap regenerates k8s/base/jcasc-configmap.yaml, then kubectl_apply runs apply -k, rollout restart, and rollout status against the controller deployment. The new pod template lands when JCasC reloads.

When the rollout completes, curl_oculus-deploy is in the job list with a Target environment dropdown showing dev plus the seven internal params (_ENVS_LIST, _ENVS_MODE, _SECRETS_JSON, _ROUTE, _ARGS_JSON, _DRY_RUN, _AGENT_LABEL).

Build it. A pod spawns on the cool-tools template — single container, no buildkitd sidecar. The run stage executes wanderland --type cli images/cool-tools/config.yml read-node --slug 'why' inside the withCredentials block, OCULUS_USER and OCULUS_PASS are set, !Env reads them, the HTTP adapter dispatches via Net::HTTP with a Base64-encoded Authorization header, and the JSON of the why node lands in the build log.

While you're here — verify the agent-pod selector

While the pod's still alive, take the inventory the Rakefile is assuming. The agent:rebuild task deletes pods matching -l jenkins=slave; newer kubernetes-plugin versions label agent pods with jenkins/label: <template-name> instead. Check yours:

kubectl -n vivarta-ci get pods --show-labels | grep cool-tools

If you only see jenkins/label=cool-tools, patch the agent:rebuild task:

sh "kubectl -n #{NS} delete pod -l jenkins/label=cool-tools --ignore-not-found=true"

Halt paths

What you have now

A small tools image, baked from vivarta-jenkins/images/cool-tools/, registered as its own pod template, built by the same containerized_deployment archetype that builds the controller and agent images. A project that targets the new template through one field (agent_label: cool-tools) and threads a basic-auth secret through withCredentials to a boundary that fetches an Oculus node via the HTTP adapter. None of the existing project-deploy machinery touched.

The boundary's HTTP adapter is mode-switchable end-to-end: live for real traffic, replay against the cassette directory you built in Step 10, mock for scenario-stubbed responses (5xx, drift, edge cases). One adapter, four modes, identical call site.

Adding a third image — aws-tools, kubectl-jq, report-builder, whatever — is the same recipe: image dir under images/, three source files, one entry in vivarta.yml's pipeline.artifacts.images, a pod template in jcasc/jenkins.yaml, an env var on the controller deployment, a project entry in projects.yml. No bash scripts.

What's missing

See also