Wanderland

Vivarta Jenkins on Kubernetes — Local Walkthrough

This is the first operational walkthrough for the Vivarta build-and-deploy front-end. Today we're going to put Jenkins on a Kubernetes cluster running locally on your Mac, and we're going to do it in a way that the same manifests carry over to CoreWeave when production is ready. Every git push (manual for now, webhooked later) will spawn a fresh Linux pod; that pod will have the Vivarta gem and the build toolchain already inside it; the pod runs vivarta deploy against the target repository; and the pod is gone the moment the build is done.

When you're done you'll be able to:

This tutorial is the operational frame for everything Vivarta does. The boundary chain that actually runs inside each pod is documented in vivarta-alb-mvp-walkthrough — read that next if you want to see what a single deploy does at the substrate level.

You should be able to work through this end to end in about two hours the first time, less than ten minutes a second time.

Before you start — what you need installed

This tutorial assumes you're on a Mac with a working terminal and a code editor. The four tools we need are OrbStack, kubectl, the Docker CLI (which ships with OrbStack), and git.

1. OrbStack. Download from https://orbstack.dev and run the installer. Open the app once so it finishes its first-run setup. OrbStack ships its own docker and kubectl binaries on PATH, so installing OrbStack covers your Docker CLI too.

2. kubectl. If which kubectl already prints a path after installing OrbStack, you're done. If not:

brew install kubectl

3. git. Installed on macOS via the Xcode Command Line Tools. If git --version errors, run xcode-select --install and re-check.

4. A text editor. Anything that handles YAML well. VS Code, Cursor, Sublime, Vim — all fine.

You don't need Java, Maven, Ruby, or any other build tool installed on your Mac. Those live inside the Vivarta agent image, not on your laptop. The only thing the laptop builds is the two Docker images, and docker build handles that.

Run all four version checks in a fresh terminal before you continue:

orb version
kubectl version --client
docker --version
git --version

If any of those four commands errors, fix that before going further.

Where we're going to run this

We have four candidate places to run real workloads, and it's worth naming each one before we pick.

Compose, locally. The fast inner loop for Vivarta development. The spring-deploy-pipeline repo already has a docker-compose.infra.yml that stands up a local Docker registry and a Reposilite Maven repo on the Mac, with Vivarta itself running as a Rack process on port 9294. There's no CI here and no ephemeral builds — this is for the boundary developer iterating on a single chain. Compose also has no native autoscaling: docker compose up --scale svc=N sets a fixed replica count and that's the whole story. We keep Compose for what it's good at, which is the fast feedback loop while we're writing Ruby.

Kubernetes, locally. This tutorial. Same Mac, but a single-node Kubernetes cluster instead of a Docker daemon. The full controller-plus-ephemeral-pod model runs here, which is what makes it the bridge between the Compose inner loop and the real production cluster. The manifests we'll write are the same manifests we'll apply to CoreWeave, with a small per-environment overlay for the things that legitimately differ (image registry, ingress, secret store).

Kubernetes on CoreWeave. The production target. The cluster is already there — iSCanGuide runs on useast13-prod, and you can see its layout in advaita-deploy/projects/iscanGuide/k8s/. When this local setup behaves correctly, porting to CoreWeave is a matter of switching kubectl context, picking up the right secret store, and changing the image registry. The shape of the Jenkins controller and the build pods stays put.

ECS with Fargate. AWS's container service in the serverless mode where you don't manage EC2 hosts. The az2-cw OAuth2 server already runs there, so it's a viable home for long-running services. Jenkins as a Fargate task works, but the ephemeral-build-pod model gets awkward: every build would be a separate task launch, with its own cold-start and its own IAM dance. We treat ECS-on-Fargate as the contingency for the Jenkins controller if the K8s path runs into a wall, and we keep an eye on it because az2 is already there.

We're picking local Kubernetes because it's the smallest step from where we are right now to where the production pipeline will live. The manifests travel, the operator's habits travel, and the runtime cost of the choice is one VM on your Mac.

A map of the moving parts

Before we touch any YAML, let's name everything that will end up running inside the cluster. There are two long-lived pieces and a swarm of short-lived ones.

The Jenkins controller is the long-lived piece. It's a single pod that runs the Jenkins web UI and the job scheduler. It carries the configured pipelines, the build history, the credentials, and the plugin set. We give it a persistent disk so a pod restart doesn't lose any of that.

The ephemeral build pods are the short-lived ones. One pod per build, gone the moment the build finishes. Each pod has three containers running side-by-side: the Jenkins agent process (automatically injected by Jenkins itself; speaks back to the controller), Vivarta and the build toolchain (where vivarta deploy actually runs), and a small image-building sidecar (used when the deploy needs to produce a new Docker image). The three containers share a localhost — they can talk to each other on 127.0.0.1 without any network plumbing.

Around those, a handful of supporting Kubernetes objects: a namespace to hold everything, two service accounts with the permissions each needs, a persistent volume for the controller's home directory, a ConfigMap that carries the controller's full configuration as a single YAML file, a Secret that carries the admin password, and two Services that expose the UI port and the controller's agent-channel port.

                ┌──────────────────────────────────────┐
                │   Jenkins controller pod             │
                │   (Deployment, 1 replica)            │
                │   ─────────────────────────────      │
                │   - JCasC ConfigMap mounted as file  │
                │   - PVC: /var/jenkins_home           │
                │   - kubernetes plugin → spawns pods  │
                └──────────────────────────────────────┘
                                │  asks the API to schedule
                                ▼
                ┌──────────────────────────────────────┐
                │  Ephemeral build pod (one per build) │
                │                                      │
                │   container: jnlp        (auto)      │
                │   container: vivarta     (work)      │
                │     - vivarta gem                    │
                │     - wanderland-core gem            │
                │     - JDK + Maven                    │
                │     - docker CLI + buildx            │
                │     - git                            │
                │   container: buildkitd   (sidecar)   │
                │     - rootless image builder         │
                └──────────────────────────────────────┘

The Jenkinsfile in the repo we're deploying stays thin — it picks the agent label, clones, and hands off:

pipeline {
  agent { label 'vivarta-agent' }
  stages {
    stage('checkout') { steps { checkout scm } }
    stage('deploy') {
      steps {
        container('vivarta') {
          sh 'vivarta deploy --config ./vivarta.yml --env dev'
        }
      }
    }
  }
}

Everything below the vivarta deploy line is the chain documented in vivarta-alb-mvp-walkthrough: boundaries that emit append-only crossings, keyed by run-id, replayable on failure.

Glossary up front

We're about to introduce about a dozen Kubernetes and Jenkins concepts. Each one gets a proper section below where it first matters. This table is the map you can hold in your head while we walk.

Term What it is
OrbStack macOS app that ships a single-node Kubernetes cluster you can start with orb start k8s.
Cluster A set of machines running Kubernetes. The local one is a single VM that pretends to be a cluster.
Namespace Subdivision of a cluster. Everything we install will live in vivarta-ci.
Pod The smallest unit Kubernetes schedules. One or more containers that share an IP and a localhost.
Deployment A controller that keeps N pods of a given shape running, replacing them if they die.
Service A stable network endpoint in front of a set of pods. ClusterIP is in-cluster only; LoadBalancer is externally reachable.
PVC Persistent Volume Claim — a request for a disk that survives pod restarts.
ConfigMap / Secret Key/value stores Kubernetes mounts into pods as files or environment variables. ConfigMaps for config; Secrets for passwords and keys.
ServiceAccount / Role / RoleBinding Identity and permissions for a pod. RBAC = Role-Based Access Control.
Jenkins controller The long-running Jenkins process. UI, scheduler, state.
Jenkins agent A worker process Jenkins spawns to run one build. We'll only use ephemeral agents.
Kubernetes plugin (for Jenkins) The plugin that lets the controller spawn agents as pods on a cluster.
JCasC Jenkins Configuration as Code — a YAML file that fully specifies the controller's configuration. We never click through the setup wizard.
podTemplate The recipe for an ephemeral build pod. Defined inside the JCasC file.
BuildKit Modern Docker-image-building engine. We run it as a sidecar so the build pod can produce images without privileged access.
Kustomize A tool that applies a stack of YAML manifests, with overlays per environment. Built into kubectl.

For a deeper conceptual reference on any Kubernetes concept, the upstream glossary at https://kubernetes.io/docs/reference/glossary/ is the authoritative place. We'll keep the explanations in this tutorial on the practical-shape side.

What OrbStack gives us

OrbStack is a macOS app you may already be running for Docker — it's the lighter-weight Docker Desktop replacement. The part we'll lean on today is less commonly used: OrbStack also ships a single-node Kubernetes cluster you can turn on from the terminal. One command, one VM, one cluster.

What we get from OrbStack that matters for this tutorial:

The Kubernetes vocabulary we'll use

A short tour of the Kubernetes objects this tutorial touches, in roughly the order we'll meet them. Each definition is small on purpose; the actual shapes show up in the numbered steps below, where we'll see them in context.

Namespace. A logical subdivision of the cluster. Everything we create — pods, services, secrets — will live in a namespace called vivarta-ci. The big practical benefit is that kubectl delete namespace vivarta-ci removes everything we've set up, in one command. Namespaces also scope names: a jenkins-controller Service in vivarta-ci doesn't conflict with one in another namespace.

Pod. The smallest unit the Kubernetes scheduler works with. A pod is one or more containers that share an IP address and a localhost. The Jenkins controller will run in a pod with one container. Each ephemeral build will run in a pod with three. The containers in a pod can address each other as 127.0.0.1:port.

Deployment. We don't create pods directly. We create a Deployment, which is a controller that says "keep one pod of this shape alive." If the pod dies, the Deployment creates a new one. If we update the Deployment, it rolls the pod over. The Jenkins controller is the only Deployment in this tutorial.

Service. A pod's IP address changes every time it restarts. A Service is a stable name and a stable IP that always points at the current pods matching a label selector. We'll create two: one for the Jenkins web UI (LoadBalancer type, externally reachable through OrbStack's networking), one for the agent channel (ClusterIP, only reachable from inside the cluster, which is where the build pods live).

PersistentVolumeClaim (PVC). Disks that survive pod restarts. Jenkins keeps its configuration, jobs, build history, and credentials under /var/jenkins_home. If we lose that directory, we lose all the state. The PVC is a request for a disk; OrbStack's storage class fulfills it automatically with a directory on the host filesystem.

ConfigMap and Secret. Both are key-value stores Kubernetes can mount into a pod as files or environment variables. The mechanical shape is the same; the only meaningful difference is that Secrets get a separate API path (so RBAC can grant access to ConfigMaps without granting it to Secrets) and a separate at-rest encryption story. Use ConfigMaps for non-sensitive config and Secrets for passwords, tokens, and keys. We'll use one ConfigMap (the JCasC file) and one Secret (the Jenkins admin password).

ServiceAccount, Role, RoleBinding. A pod runs under an identity called a ServiceAccount. A Role names a set of permissions inside one namespace (verbs like get, list, create against resource kinds like pods or secrets). A RoleBinding ties a ServiceAccount to a Role. Together, these three are called RBAC — Role-Based Access Control. The Jenkins controller needs RBAC to create and delete build pods on its own behalf; without it, the controller could not spawn agents at all.

How Jenkins separates the controller from the agents

Jenkins splits the world into two roles, and the split matters because it's what makes the ephemeral-pod pattern possible.

The controller is the long-running Jenkins process — the web UI, the job scheduler, the credentials store, the build history, the plugin set. The controller is supposed to be small, well-behaved, and always running. It schedules builds, but it does not execute them.

The agent is the worker. It actually runs the build's shell commands. Agents historically ran on separate machines you provisioned ahead of time; the Jenkins controller would SSH into them and start a process. We're not going to do that.

We want ephemeral agents — agents that come into existence the moment a build needs one, run that one build, and disappear. The mechanism that makes this work in Kubernetes is a plugin.

The Kubernetes plugin and what podTemplate means

The Kubernetes plugin for Jenkins teaches the controller to schedule its agents as Kubernetes pods. When a build is queued, the plugin uses the cluster's API to create a new pod that matches a podTemplate. The pod starts, the agent inside it connects back to the controller, the build runs, the pod is deleted. Next build, fresh pod.

The podTemplate is a piece of YAML the controller carries — it names the agent's label (which is how the Jenkinsfile picks the right kind of agent), the namespace the pod lives in, the service account the pod runs under, and the list of containers the pod will have. The Jenkins agent process itself (the container called jnlp) is auto-injected by the plugin — we don't declare it. We declare the work containers: in our case, one called vivarta carrying the Vivarta gem and the toolchain, and one called buildkitd for image builds.

We will define the podTemplate inside the JCasC file. That keeps every operational fact about how the controller behaves in one place.

Why we configure Jenkins from a file (JCasC)

The default Jenkins experience is "open the web UI and click through a setup wizard, then click around to add credentials and plugins and clouds." That works for a single human, once, and it produces a configuration that lives in /var/jenkins_home as a tangle of XML files. There is no version history. There is no review process. If we lose the pod, we lose the configuration unless we remembered to back up the volume.

Jenkins Configuration as Code (JCasC) is a plugin that loads the full controller configuration from a YAML file at startup. Security realm, authorization strategy, clouds, credentials, system message, executors, podTemplates — all of it. When the file is checked into git and mounted into the pod as a ConfigMap, the configuration becomes a thing we can read, review, and roll forward like any other code.

We're going to:

There is no other path through this tutorial. We never log into the UI to change a setting.

Why we build images with BuildKit, not Docker-in-Docker

The Vivarta deploy needs to produce Docker images — that's what compile_jar → docker_build → compose_deploy does today, and the same will be true under K8s. Building a Docker image inside a Kubernetes pod has three common patterns, and the difference between them is mostly about how much privilege you grant the build container.

Docker-in-Docker (DinD) runs a full Docker daemon inside the build pod. The build container talks to the in-pod daemon over a socket. DinD works on every platform, but it requires the container to run with --privileged, which gives it broad access to the host kernel. It's the legacy choice and the one most older tutorials show.

Kaniko was Google's response to DinD: an image builder that reads a Dockerfile and produces an image without needing any daemon at all. Kaniko was archived in January 2025. We're not building on a dead project.

BuildKit is the engine Docker itself moved to. We run buildkitd as a rootless sidecar container in the build pod, listening on localhost:1234 inside the pod. The vivarta container runs docker buildx which talks to the sidecar over loopback. No privileged container, no host socket mount, and the same approach works identically on OrbStack and on CoreWeave.

That's what we'll wire into the podTemplate below.

Kustomize, briefly

kubectl apply -k <directory> applies a directory of YAML manifests as a unit. The directory has a kustomization.yaml that lists the files and lets us layer per-environment overrides on top of a shared base. We won't lean hard on Kustomize in this tutorial — one base, the OrbStack overlay we're running today, and a CoreWeave overlay we'll point at later — but we'll set up the directory structure so the CoreWeave move stays small.

Repository layout we're going to build

We're going to create a new directory called vivarta-jenkins. By the end of the tutorial it looks like this:

vivarta-jenkins/
  .gitignore
  images/
    controller/
      Dockerfile              — Jenkins LTS + the plugin set baked in
      plugins.txt
    agent/
      Dockerfile              — JDK + Maven + Ruby + vivarta + buildx
      bin/vivarta             — thin shell wrapper
  jcasc/
    jenkins.yaml              — JCasC config (security, kubernetes cloud, podTemplate)
  k8s/
    base/
      namespace.yaml
      rbac.yaml
      pvc.yaml
      secret.yaml.example     — admin password (copy and fill in; do not commit secret.yaml)
      jcasc-configmap.yaml    — generated from jcasc/jenkins.yaml
      deployment.yaml
      service.yaml
      kustomization.yaml
    overlays/
      orbstack/               — what we're using today
      coreweave/              — what we'll switch to later

The Kustomize shape mirrors what advaita-deploy/projects/iscanGuide/k8s/ already uses for the iSCanGuide production deployment. Same conventions; same overlay pattern.

0. Create the project directory

Pick a parent directory (we'll use ~/working/) and create the project layout:

mkdir -p ~/working/vivarta-jenkins
cd ~/working/vivarta-jenkins
mkdir -p images/controller images/agent/bin jcasc k8s/base k8s/overlays/orbstack k8s/overlays/coreweave
git init

Create .gitignore to keep the admin Secret out of source control:

cat > .gitignore <<'EOF'
# Local-only secrets; the .example version is in git.
k8s/base/secret.yaml

# Generated from jcasc/jenkins.yaml; regenerated by the bring-up command.
# Tracking it in git is fine if you prefer the audit trail; ignore if you don't.
k8s/base/jcasc-configmap.yaml
EOF

Verify the shape:

ls -la
# expect: .git, .gitignore, images/, jcasc/, k8s/

tree
.
├── images
│   ├── agent
│   │   └── bin
│   └── controller
├── jcasc
└── k8s
    ├── base
    └── overlays
        ├── coreweave
        └── orbstack

11 directories, 0 files

Every file we create from now on goes inside this directory.

1. Start the cluster and create the namespace

Turn on OrbStack's Kubernetes if it isn't already running:

orb start k8s

The first time, OrbStack pulls images and provisions the VM (about a minute on a recent Mac). After that, starting the cluster takes under a second.

Confirm kubectl is pointed at the right cluster:

kubectl config use-context orbstack
kubectl get nodes

You should see a single node with status Ready. If kubectl get nodes errors with "couldn't get current server API group list" or similar, OrbStack hasn't finished starting yet — wait a few seconds and retry.

Now create the namespace. Put this in k8s/base/namespace.yaml:

# k8s/base/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: vivarta-ci

Apply it:

kubectl apply -f k8s/base/namespace.yaml

Verify:

kubectl get namespace vivarta-ci
# NAME         STATUS   AGE
# vivarta-ci   Active   5s

2. Give the controller permission to spawn pods

The Kubernetes plugin will ask the API to create pods on our behalf. For that, the controller pod needs to run under a ServiceAccount with the right permissions. The build pods run under a separate ServiceAccount with effectively no permissions — they don't need to talk to the API at all.

Put this in k8s/base/rbac.yaml:

# k8s/base/rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata: { name: jenkins-controller, namespace: vivarta-ci }
---
apiVersion: v1
kind: ServiceAccount
metadata: { name: jenkins-agent, namespace: vivarta-ci }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: jenkins-controller, namespace: vivarta-ci }
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/exec", "pods/log"]
    verbs: ["get", "list", "watch", "create", "delete", "deletecollection"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: { name: jenkins-controller, namespace: vivarta-ci }
roleRef: { apiGroup: rbac.authorization.k8s.io, kind: Role, name: jenkins-controller }
subjects:
  - kind: ServiceAccount
    name: jenkins-controller
    namespace: vivarta-ci

Anatomy of the four objects in that file:

Apply and verify:

kubectl apply -f k8s/base/rbac.yaml

kubectl -n vivarta-ci get serviceaccount,role,rolebinding
# NAME                                   SECRETS   AGE
# serviceaccount/default                 0         12m
# serviceaccount/jenkins-controller      0         3s
# serviceaccount/jenkins-agent           0         3s
#
# NAME                                          CREATED AT
# role.rbac.authorization.k8s.io/jenkins-controller   ...
#
# NAME                                                 ROLE                       AGE
# rolebinding.rbac.authorization.k8s.io/jenkins-controller   Role/jenkins-controller    3s

3. The Jenkins admin password as a Secret

We're going to give JCasC a username and a password through environment variables. The values come from a Kubernetes Secret.

Create the example template at k8s/base/secret.yaml.example. This file goes into git:

# k8s/base/secret.yaml.example  →  copy to secret.yaml and edit
apiVersion: v1
kind: Secret
metadata: { name: jenkins-admin, namespace: vivarta-ci }
type: Opaque
stringData:
  JENKINS_ADMIN_ID: admin
  JENKINS_ADMIN_PASSWORD: change-me-strong-password

Two practical notes:

Copy the example, edit it with a real password, and apply:

cp k8s/base/secret.yaml.example k8s/base/secret.yaml

# Edit the password in your editor; replace 'change-me-strong-password'
# with something only you know. Save and close.
$EDITOR k8s/base/secret.yaml

kubectl apply -f k8s/base/secret.yaml

Verify:

kubectl -n vivarta-ci get secret jenkins-admin
# NAME            TYPE     DATA   AGE
# jenkins-admin   Opaque   2      3s

DATA 2 confirms both keys (JENKINS_ADMIN_ID and JENKINS_ADMIN_PASSWORD) made it in.

4. Build the Jenkins controller image

"The controller image" — let's pause and unpack what we mean. The Jenkins controller is a pod, and a pod runs a container, and a container starts from an image. Upstream Jenkins publishes an official image called jenkins/jenkins:lts-jdk21 that you can run as-is. The trouble is that it ships with zero plugins. The first time you boot a vanilla controller, you sit at the UI waiting for it to download Kubernetes plugin, JCasC plugin, Pipeline plugin, Git plugin — a long list — over the network. Every restart is another network-dependent boot.

We solve that by building our own controller image on top of the official one: same Jenkins, but with the plugins we need already installed. The plugin list becomes part of the image and part of git. We rebuild the image when we want to change it.

Create the plugin list at images/controller/plugins.txt:

configuration-as-code
kubernetes
kubernetes-credentials-provider
workflow-aggregator
git
github-branch-source
job-dsl
credentials-binding
matrix-auth
pipeline-utility-steps
timestamper
ws-cleanup

What each plugin gives us:

Create the Dockerfile at images/controller/Dockerfile:

# images/controller/Dockerfile
FROM jenkins/jenkins:lts-jdk21

USER root
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt
USER jenkins

# Skip the setup wizard so JCasC can take over on first boot.
ENV JAVA_OPTS="-Djenkins.install.runSetupWizard=false"
ENV CASC_JENKINS_CONFIG=/var/jenkins_home/casc_configs

Anatomy:

Build it:

docker build -t vivarta-jenkins-controller:0.1.0 images/controller

The first build takes about a minute — most of that is the plugin-cli downloading plugin JARs. The result is in your local Docker image store; OrbStack puts that same tag in front of the cluster so pods can pull it without a registry.

Verify:

docker image ls vivarta-jenkins-controller
# REPOSITORY                   TAG     IMAGE ID       CREATED         SIZE
# vivarta-jenkins-controller   0.1.0   ...            30 seconds ago  ~700MB

# Sanity-check that the plugins actually landed in the image.
docker run --rm vivarta-jenkins-controller:0.1.0 ls /usr/share/jenkins/ref/plugins | head
# configuration-as-code.jpi
# kubernetes.jpi
# ... eleven more

If that ls returns nothing, the plugins.txt file in your build context is empty. docker build doesn't error on an empty file — jenkins-plugin-cli runs against nothing and exits cleanly, and the controller boots without any plugins, which makes JCasC silently no-op because none of its YAML keys map to known plugin types. Save the file before you build.

5. Build the Vivarta build-agent image

This is the more interesting image. It's what every ephemeral build pod will run as its main vivarta container. We bake in everything vivarta deploy needs to do its job: the JDK and Maven (for compiling Java code), Ruby and the Vivarta gem (for running the deploy chain), the Docker CLI plus buildx (for talking to the BuildKit sidecar when an image build is needed), and git (for any further checkouts the chain itself needs).

The agent image needs four sibling repositories on disk — Vivarta itself plus the three Wanderland gems it depends on. The Vivarta Gemfile declares them all as path-deps:

gem "wanderland-core",           path: "../wanderland-core"
gem "wanderland-aws-pack",       path: "../wanderland-aws-pack"
gem "wanderland-pipelines-pack", path: "../wanderland-pipelines-pack"

We expect all four to be checked out under your working directory as siblings (~/working/vivarta, ~/working/wanderland-core, ~/working/wanderland-aws-pack, ~/working/wanderland-pipelines-pack). If any are missing, clone them now.

Rather than copying or symlinking the source repos into the build context, we use BuildKit's --build-context flag at build time. Each --build-context name=path tells BuildKit "this name maps to that directory; the Dockerfile can COPY --from=<name> from it." No symlinks, no parent-directory builds, no exclusions: explicit named inputs.

Create images/agent/Dockerfile:

# images/agent/Dockerfile
FROM eclipse-temurin:17-jdk-noble

# Build toolchain. Noble (Ubuntu 24.04) ships Ruby 3.2 by default, which
# satisfies the Wanderland gem stack's >= 3.1 requirement.
RUN apt-get update && apt-get install -y --no-install-recommends \
      git curl ca-certificates \
      maven \
      ruby ruby-dev build-essential \
      docker.io docker-buildx \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /opt/vivarta

# Each --build-context flag on the docker build command line maps a name
# to a directory on the host; COPY --from=<name> reads from it.
COPY --from=vivarta                   . /opt/vivarta/
COPY --from=wanderland-core           . /opt/wanderland-core/
COPY --from=wanderland-aws-pack       . /opt/wanderland-aws-pack/
COPY --from=wanderland-pipelines-pack . /opt/wanderland-pipelines-pack/

RUN gem install bundler \
 && cd /opt/vivarta \
 && bundle config set --local without 'development test' \
 && bundle install

# `vivarta` on PATH points at wanderland-core's CLI adapter, pre-bound to
# the Vivarta config. See "The CLI entry point" below.
COPY bin/vivarta /usr/local/bin/vivarta
RUN chmod +x /usr/local/bin/vivarta

WORKDIR /workspace

Anatomy:

The CLI entry point

wanderland-core ships a binary called wanderland that takes a --type switch (http, cli, or test) and a config path. --type cli reads the config's routes: block and generates a Commander-style subcommand for every route — so wanderland --type cli config.yml deploy runs the same chain that POST /deploy would, but synchronously from the shell, with the run's terminal envelope printed to stdout.

A two-line vivarta shim binds the type and config path so the Jenkinsfile stays readable.

Create images/agent/bin/vivarta:

#!/usr/bin/env bash
exec bundle exec --gemfile=/opt/vivarta/Gemfile \
  wanderland --type cli /opt/vivarta/config.yml "$@"

Anatomy:

Build the image. Run this from your vivarta-jenkins directory so the relative paths resolve to the sibling repos:

docker build -t vivarta-agent:0.1.0 \
  --build-context vivarta=../vivarta \
  --build-context wanderland-core=../wanderland-core \
  --build-context wanderland-aws-pack=../wanderland-aws-pack \
  --build-context wanderland-pipelines-pack=../wanderland-pipelines-pack \
  images/agent

First build is two to three minutes — mostly apt-get install and bundle install.

Verify the image and the CLI:

docker image ls vivarta-agent
# REPOSITORY      TAG     IMAGE ID       CREATED          SIZE
# vivarta-agent   0.1.0   ...            1 minute ago     ~1.5GB

# `vivarta help` lists every route as a subcommand.
docker run --rm vivarta-agent:0.1.0 vivarta help
# COMMANDS:
#   deploy        Route POST /deploy
#   health        Route GET  /health
#   healthcheck   Route GET  /healthcheck
#   inspect ...   Route GET  /inspect/...
#   status        Route GET  /status

vivarta help <command> prints the per-route options. vivarta deploy runs the deploy chain end-to-end against the in-memory adapter mode — useful as a local smoke test before the pipeline runs it in a real build pod.

6. Write the JCasC file

This is the heart of the controller. Every operational fact about how Jenkins behaves — who can log in, where credentials come from, what Kubernetes cloud it can spawn pods on, what those pods look like — lives in this one file.

Create jcasc/jenkins.yaml:

# jcasc/jenkins.yaml
jenkins:
  systemMessage: "Vivarta CI — local OrbStack"
  numExecutors: 0          # the controller does no work; every build runs on an agent
  mode: EXCLUSIVE          # only jobs that explicitly request a label run
  securityRealm:
    local:
      allowsSignup: false
      users:
        - id: "${JENKINS_ADMIN_ID}"
          password: "${JENKINS_ADMIN_PASSWORD}"
  authorizationStrategy:
    loggedInUsersCanDoAnything:
      allowAnonymousRead: false

  clouds:
    - kubernetes:
        name: "vivarta-k8s"
        serverUrl: "https://kubernetes.default.svc"
        namespace: "vivarta-ci"
        jenkinsUrl: "http://jenkins-controller.vivarta-ci.svc.cluster.local:8080"
        jenkinsTunnel: "jenkins-agent.vivarta-ci.svc.cluster.local:50000"
        connectTimeout: 30
        readTimeout: 60
        templates:
          - name: "vivarta-agent"
            namespace: "vivarta-ci"
            label: "vivarta-agent"
            serviceAccount: "jenkins-agent"
            yaml: |
              apiVersion: v1
              kind: Pod
              spec:
                containers:
                  - name: vivarta
                    image: vivarta-agent:0.1.0
                    imagePullPolicy: IfNotPresent
                    command: ["sleep"]
                    args: ["999999"]
                    tty: true
                    env:
                      - name: BUILDKIT_HOST
                        value: tcp://localhost:1234
                      - name: DOCKER_BUILDKIT
                        value: "1"
                    resources:
                      requests: { cpu: "500m", memory: "1Gi" }
                      limits:   { cpu: "2",    memory: "4Gi" }
                  - name: buildkitd
                    image: moby/buildkit:rootless
                    args:
                      - --addr
                      - tcp://0.0.0.0:1234
                      - --oci-worker-no-process-sandbox
                    securityContext:
                      seccompProfile: { type: Unconfined }
                      runAsUser: 1000
                      runAsGroup: 1000
                    resources:
                      requests: { cpu: "500m", memory: "1Gi" }
                      limits:   { cpu: "2",    memory: "4Gi" }

Anatomy of the top half (jenkins: block):

Anatomy of the clouds.kubernetes block:

Anatomy of the podTemplate:

7. Wrap JCasC in a ConfigMap

The JCasC file is a YAML file on your laptop. Kubernetes mounts it into the controller pod through a ConfigMap. We keep the source of truth at jcasc/jenkins.yaml and regenerate the ConfigMap manifest from it whenever it changes.

Generate the ConfigMap manifest at k8s/base/jcasc-configmap.yaml:

kubectl create configmap jenkins-jcasc \
  --namespace vivarta-ci \
  --from-file=jenkins.yaml=jcasc/jenkins.yaml \
  --dry-run=client -o yaml > k8s/base/jcasc-configmap.yaml

What that command does:

Run that command any time you edit jcasc/jenkins.yaml. Step 12 will rerun it as part of the bring-up.

8. Persistent storage for the controller

Create k8s/base/pvc.yaml:

# k8s/base/pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata: { name: jenkins-home, namespace: vivarta-ci }
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 10Gi

Anatomy:

9. The Jenkins controller Deployment

Create k8s/base/deployment.yaml:

# k8s/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: jenkins-controller, namespace: vivarta-ci }
spec:
  replicas: 1
  strategy: { type: Recreate }       # the PVC is RWO; a rolling update would deadlock
  selector:
    matchLabels: { app: jenkins-controller }
  template:
    metadata:
      labels: { app: jenkins-controller }
    spec:
      serviceAccountName: jenkins-controller
      securityContext:
        fsGroup: 1000                 # let the jenkins user write to the mounted PVC
      containers:
        - name: jenkins
          image: vivarta-jenkins-controller:0.1.0
          imagePullPolicy: IfNotPresent
          ports:
            - { name: http, containerPort: 8080 }
            - { name: jnlp, containerPort: 50000 }
          envFrom:
            - secretRef: { name: jenkins-admin }
          env:
            - name: CASC_JENKINS_CONFIG
              value: /var/jenkins_home/casc_configs
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
            - name: jcasc
              mountPath: /var/jenkins_home/casc_configs
              readOnly: true
          readinessProbe:
            httpGet: { path: /login, port: http }
            initialDelaySeconds: 30
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /login, port: http }
            initialDelaySeconds: 90
            periodSeconds: 30
      volumes:
        - name: jenkins-home
          persistentVolumeClaim: { claimName: jenkins-home }
        - name: jcasc
          configMap: { name: jenkins-jcasc }

Anatomy:

10. Two Services — one for you, one for the build pods

Create k8s/base/service.yaml:

# k8s/base/service.yaml
apiVersion: v1
kind: Service
metadata: { name: jenkins-controller, namespace: vivarta-ci }
spec:
  type: LoadBalancer
  selector: { app: jenkins-controller }
  ports:
    - { name: http, port: 8080, targetPort: 8080 }
---
apiVersion: v1
kind: Service
metadata: { name: jenkins-agent, namespace: vivarta-ci }
spec:
  type: ClusterIP
  selector: { app: jenkins-controller }
  ports:
    - { name: jnlp, port: 50000, targetPort: 50000 }

Anatomy:

11. The Kustomization manifest

Create k8s/base/kustomization.yaml:

# k8s/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: vivarta-ci
resources:
  - namespace.yaml
  - rbac.yaml
  - pvc.yaml
  - secret.yaml
  - jcasc-configmap.yaml
  - deployment.yaml
  - service.yaml

kubectl apply -k k8s/base will read this file and apply every listed manifest in order. The namespace: vivarta-ci line at the top sets the default namespace for every resource, so individual manifests don't have to repeat it.

12. Bring it all up

We've created all the files. Time to apply the stack and watch Jenkins boot.

Regenerate the ConfigMap (in case jcasc/jenkins.yaml changed since step 7) and apply everything:

kubectl create configmap jenkins-jcasc \
  --namespace vivarta-ci \
  --from-file=jenkins.yaml=jcasc/jenkins.yaml \
  --dry-run=client -o yaml > k8s/base/jcasc-configmap.yaml

kubectl apply -k k8s/base

Watch the controller pod come up:

kubectl -n vivarta-ci get pods -w

Expected sequence (Ctrl-C out of -w once you see Running 1/1):

NAME                                  READY   STATUS              RESTARTS   AGE
jenkins-controller-xxxx-xxxx          0/1     Pending             0          1s
jenkins-controller-xxxx-xxxx          0/1     ContainerCreating   0          2s
jenkins-controller-xxxx-xxxx          0/1     Running             0          15s
jenkins-controller-xxxx-xxxx          1/1     Running             0          45s   # ← Ready

The pod becomes Ready once the readiness probe (/login on port 8080) passes — that's around 45–60 seconds after the container starts on a first boot.

Get the LoadBalancer's address:

kubectl -n vivarta-ci get svc jenkins-controller
# NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)
# jenkins-controller   LoadBalancer   10.96.x.x       198.19.249.x    8080:.../TCP

The EXTERNAL-IP column is what you'll put in your browser. Open http://<EXTERNAL-IP>:8080.

Log in with the username and password from k8s/base/secret.yaml (defaults: admin and whatever you set the password to). You should land on the Jenkins dashboard with no setup wizard — JCasC took care of that. The Kubernetes cloud is configured, the vivarta-agent podTemplate is registered, and you're ready to create a job.

If anything looks wrong:

# Tail the controller's logs
kubectl -n vivarta-ci logs deploy/jenkins-controller -f

# Describe the pod for events (image-pull errors, mount failures, etc.)
kubectl -n vivarta-ci describe pod -l app=jenkins-controller

Any time you change jcasc/jenkins.yaml, regenerate and reload:

kubectl create configmap jenkins-jcasc \
  --namespace vivarta-ci \
  --from-file=jenkins.yaml=jcasc/jenkins.yaml \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl -n vivarta-ci rollout restart deployment/jenkins-controller

13. Your first build — proving the chain end-to-end

In the Jenkins UI, create a Pipeline job pointing at the hello-spring repository:

pipeline {
  agent { label 'vivarta-agent' }
  environment {
    VIVARTA_RUN_ID = "${env.JOB_NAME}-${env.BUILD_NUMBER}"
  }
  stages {
    stage('checkout') {
      steps {
        // For first-pass testing without GitHub auth wired, clone over HTTPS.
        // Replace with a real repo URL.
        git url: 'https://github.com/your-org/hello-spring.git', branch: 'main'
      }
    }
    stage('deploy') {
      steps {
        container('vivarta') {
          sh 'vivarta deploy --config ./vivarta.yml --env dev'
        }
      }
    }
  }
}

Watch what happens, from your shell:

# A new pod gets scheduled the moment the build starts
kubectl -n vivarta-ci get pods -l jenkins/label=vivarta-agent -w

# Each build pod has three containers: jnlp + vivarta + buildkitd
kubectl -n vivarta-ci describe pod <pod-name>

# Watch vivarta's stdout in real time
kubectl -n vivarta-ci logs <pod-name> -c vivarta -f

# When the build ends, the pod is gone within a few seconds

Each pod is single-use. A second build gets a second pod. There's no shared mutable state between builds — every run starts from the same bytes, and the run-id is the only thing that distinguishes them. That property is what the vivarta-alb-mvp-walkthrough replay model depends on.

Replay model — what the substrate gets out of this

The append-only crossing substrate plus run-id-keyed addressing means a failed Jenkins build can be re-run from scratch with the same VIVARTA_RUN_ID, and each boundary skips the work it already completed (by checking its (from_addr, type_addr) signature against existing crossings). The Jenkinsfile we wrote in step 13 already sets that ID consistently:

environment {
  // Stable across retries of the same Jenkins build number.
  VIVARTA_RUN_ID = "${env.JOB_NAME}-${env.BUILD_NUMBER}"
}

Jenkins's Replay feature reruns the Jenkinsfile against the same BUILD_NUMBER, which means the same run-id, which means Vivarta's per-boundary idempotency check fires and skips the boundaries that already wrote their *-complete crossings. The skip mechanism is fully documented in vivarta-alb-mvp-walkthrough.

Porting to CoreWeave

Three things change between the OrbStack overlay and the CoreWeave overlay; the base is unchanged.

Concern OrbStack (today) CoreWeave (later)
Image source Local Docker daemon (shared engine) ECR or registry — imagePullSecrets referenced from each ServiceAccount
External access Service: type: LoadBalancer resolves to a host IP Service: type: LoadBalancer resolves to a CoreWeave-assigned external IP. We add an Ingress for FQDN plus TLS once cert-manager is in.
Secrets Inline Secret (gitignored) External Secrets Operator → AWS Secrets Manager or Vault. iSCanGuide's prod overlay already uses this pattern.

The manifests that don't change: namespace, rbac, pvc, deployment, the JCasC ConfigMap, the podTemplate inside it, the BuildKit sidecar pattern. The Vivarta agent image is the same image, pushed to ECR once, pulled in both clusters.

The kubeconfig switch is one line:

kubectl config use-context useast13-prod
kubectl apply -k k8s/overlays/coreweave

useast13-prod is the same context iSCanGuide already uses. See advaita-deploy/projects/iscanGuide/k8s/overlays/prod/ for the namespace-isolation, resource-quota, and external-secrets patterns to mirror.

Tearing it all down

When you're done experimenting and want to start over:

# Drops everything in vivarta-ci — pods, services, secrets, PVC.
kubectl delete namespace vivarta-ci

# Stop the local cluster (optional — OrbStack restarts in under a second).
orb stop k8s

To bring it back up, re-run step 1 forward. Your jcasc/jenkins.yaml, your Dockerfiles, and your manifests all survive — they live in your vivarta-jenkins directory.

What this tutorial doesn't cover yet

What's next