Wanderland

Vivarta Jenkins Tutorial: CFN Verify Tool

Third tutorial in the vivarta-on-k8s series. After the parent tutorial (controller + agent) and the secrets tutorial (cool-tools image with credentials), this one teaches composition — bundling several boundaries into a single tool the user invokes once.

The worked example is CloudFormation verify: render parameters from a central config, validate the template against AWS, lint with cfn-lint, simulate the change set. Four boundaries, one wrapper, one route. Run it before any real deploy and find out what would happen.

The image is called cfn-tools — five Ruby boundaries in vivarta-jenkins/images/cfn-tools/, plus the same Dockerfile + config.yml + build-script trio as the secrets tutorial.

When you're done you'll be able to:

About forty minutes the first time.

Where this fits

There are two ways to expose a multi-step pipeline as a tool:

Spell the chain out in the route. A route's chain: lists every boundary in order with its args. Maximum visibility — anyone reading the config sees exactly what runs. Brittle as the chain grows; every consumer repeats the same five-line list.

Wrap the chain in one boundary. What we're doing. The wrapper is a Ruby class that dispatches its sub-boundaries in sequence, halts on the first failure, aggregates results at the end. Route declares one boundary; the boundary's contract names what it needs and what it returns. Same pattern ecs_service_deploy in wanderland-aws-pack uses for register-task → poke-service → wait-stable.

Wrapping is the right call when the chain is stable, has internal data flow, and presents as one user-visible thing.

What we're going to build

vivarta-jenkins/images/cfn-tools/
  Dockerfile
  config.yml
  lib/cfn_tools/boundaries/
    cfn_verify.rb              # the wrapper — dispatches the four below
    render_params.rb           # substitute config.yaml values into params
    cfn_validate_template.rb   # AWS SDK validate_template
    cfn_lint_template.rb       # shells out to cfn-lint
    cfn_change_set_preview.rb  # AWS SDK create + describe + delete change set

Plus one entry in vivarta.yml's pipeline.artifacts.images for the image build, a third pod-template entry in jcasc/jenkins.yaml, a CFN_TOOLS_IMAGE_REF env var on the controller deployment, and a project entry pointing at it.

Six boundary files, one wrapper, one route, one pipeline entry.

What the AWS adapter gives us

The :aws adapter kind (from wanderland-aws-pack) wraps Aws::*::Client for any service. We declare it in config.yml against service: cloudformation:

adapters:
  cloudformation:
    kind: aws
    service: cloudformation
    region: us-west-2
    mode: !Env { name: WANDERLAND_AWS_MODE, default: live }
    cassette_dir: cassettes

The boundary pulls the configured instance via input.adapter(:cloudformation) at call time. Real calls in live mode, recorded responses in replay, scenario-provided responses in mock, synthesized SDK shapes in dry-run. Same shape across all four modes — the boundary code doesn't branch on mode.

First live run produces cassettes when --wanderland-record-to <dir> is on the request. Subsequent replay/mock runs use them.

Step 0. Prerequisites

Step 1. Clone the repo

git clone git@github
cd vivarta-jenkins

(Or git pull an existing checkout from the parent tutorials.)

Step 2. Create the image directory

mkdir -p images/cfn-tools/lib/cfn_tools/boundaries

Six files go in there — top-level Dockerfile + config.yml, four boundary .rb files plus the wrapper.

Step 3. The wrapper boundary

cfn_verify is one boundary that runs four others. Users invoke it; the four sub-boundaries are implementation detail. The framework's chain runner + dispatch helper handle the plumbing — sub-crossings accumulate in the wrapper's inner context, the rollup combines their payloads via last-writer-wins into the wrapper's outer crossing, and an _signals audit trail rides along on the result payload so the inner trace stays recoverable.

There are two ways to express the wrapper: the declarative chain: form (no #call body, framework runs the chain), and the imperative #call form using the dispatch() helper for cases that need conditional logic between sub-dispatches. cfn_verify wants the imperative form because the lint step's threshold check sits between two halt-checks.

# images/cfn-tools/lib/cfn_tools/boundaries/cfn_verify.rb
module CfnTools
  module Boundaries
    class CfnVerify
      include Wanderland::Boundary

      IDENTITY = Wanderland::Identity.new(
        id:     "boundary:cfn_verify",
        name:   "CfnVerify",
        roles:  [:boundary],
        type:   :service,
        scopes: %i[execute read]
      ).freeze

      boundary :cfn_verify,
        identity:     IDENTITY,
        requirements: %i[execute read],
        capabilities: [:verify],
        description:  "Validate CFN template + params, surface what AWS would create",
        input_shape: {
          "args" => {
            "template_file" => {
              required:    true,
              description: "Path to the CloudFormation template YAML to verify"
            },
            "params_file" => {
              required:    true,
              description: "Path to the params YAML — REPLACE_ME slots resolve from config_file"
            },
            "config_file" => {
              required:    true,
              description: "Path to the central config YAML supplying real values for REPLACE_ME slots"
            },
            "stack_name" => {
              required:    true,
              description: "Stack name AWS would create — surfaces in the change-set name",
              constraints: { matches: "^[a-z][a-z0-9-]{2,127}$" }
            },
            "lint_threshold" => {
              required:    false,
              description: "Severity floor that halts the verify chain (info | warning | error)",
              constraints: { includes: %w[info warning error] },
              metadata:    { default_visible_to_operator: "warning" }
            }
          }
        },
        output_shape: {
          "parameters_resolved" => true,
          "template_valid"      => true,
          "lint_findings"       => true,
          "lint_count"          => true,
          "resources_to_create" => true
        },
        adapters: %i[cloudformation cfn_lint]

      def call(input)
        threshold = input.find("args.lint_threshold") || "warning"

        # render_params emits `parameters` into the wrapper's inner
        # context. cfn_validate_template halts on bad syntax; the
        # halt rethrows through dispatch and surfaces as the wrapper's
        # halt — no manual passthrough plumbing.
        input.dispatch(:render_params)
        input.dispatch(:cfn_validate_template)

        # cfn_lint_template's findings flow through unless severity
        # meets `threshold`. We capture its result hash directly to
        # inspect the findings before deciding whether to halt.
        lint = input.dispatch(:cfn_lint_template)
        if lint_blocks?(lint["findings"], threshold)
          return Wanderland::Signal.halt(
            status:    422,
            error:     "cfn-lint findings meet or exceed threshold #{threshold.inspect}",
            findings:  lint["findings"]
          )
        end

        # cfn_change_set_preview needs the resolved parameters from
        # step 1's emission. `project("context.parameters")` reads
        # from the wrapper's inner context, which has render_params'
        # crossing in it by the time we get here. The framework
        # resolves the projection before the sub sees its args.
        input.dispatch(:cfn_change_set_preview,
          parameters_resolved: Wanderland::Projection.new("context.parameters"))

        # Everything the subs emitted (parameters, template_valid,
        # lint_findings, lint_count, resources_to_create, change_set_id)
        # LWW-merges into the wrapper's outgoing payload via the
        # framework's rollup. `_signals` carries the four inner
        # crossings as an audit trail on the same payload.
        Wanderland::Signal.ok
      end

      private

      def lint_blocks?(findings, threshold)
        return false unless findings.is_a?(Array)
        levels = { "info" => 0, "warning" => 1, "error" => 2 }
        bar = levels[threshold.to_s] || 1
        findings.any? { |f| (levels[f["level"].to_s] || 0) >= bar }
      end
    end
  end
end

Anatomy:

Step 4. The sub-boundaries

Four small files. Each does one thing. These will eventually live in wanderland-pipelines-pack (render_params) and wanderland-aws-pack (the cfn_* ones); the tutorial mounts them locally so you can read the whole composition in one place.

Each sub-boundary is just a regular boundary — no awareness of being part of a wrapper. It reads its args, talks to its adapter (or runs as a pure transform), and emits a Signal. When cfn_verify dispatches one, the sub's crossing lands in the wrapper's inner context (not the outer route context), and the framework's rollup picks it up to merge into cfn_verify's payload. A sub-boundary that halts produces a :signals:stop:* crossing; input.dispatch re-throws it inside the wrapper's #call, the registration's catch fields it, the rollup propagates the halt as the wrapper's own.

Step 5. The config

# images/cfn-tools/config.yml
service: cfn-tools
boundary_path: lib/cfn_tools/boundaries

adapters:
  cloudformation:
    kind: aws
    service: cloudformation
    region: !Env { name: AWS_REGION, default: us-west-2 }
    mode:   !Env { name: WANDERLAND_AWS_MODE, default: live }
    cassette_dir: cassettes

  cfn_lint:
    kind: shell
    command: cfn-lint
    mode: !Env { name: WANDERLAND_AWS_MODE, default: live }
    cassette_dir: cassettes
    baseline_env:
      PATH: !Env { name: PATH, default: "" }

routes:
  /verify/:stack_name:
    method: post
    boundary: cfn_verify
    name: verify
    args:
      stack_name:     !PathCapture stack_name
      template_file:  !Environment CFN_TEMPLATE
      params_file:    !Environment CFN_PARAMS
      config_file:    !Environment CFN_CONFIG
      lint_threshold: !Environment CFN_LINT_THRESHOLD

Anatomy:

Step 6. The Dockerfile

# images/cfn-tools/Dockerfile
FROM ruby:3.4-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
      build-essential ca-certificates git python3-pip \
    && pip3 install --break-system-packages cfn-lint \
    && rm -rf /var/lib/apt/lists/*

# wanderland-core + wanderland-aws-pack arrive via --build-context.
COPY --from=wanderland-core . /tmp/wanderland-core/
RUN cd /tmp/wanderland-core && gem build wanderland-core.gemspec \
    && gem install --local wanderland-core-*.gem && rm -rf /tmp/wanderland-core

COPY --from=wanderland-aws-pack . /tmp/wanderland-aws-pack/
RUN cd /tmp/wanderland-aws-pack && gem build wanderland-aws-pack.gemspec \
    && gem install --local wanderland-aws-pack-*.gem && rm -rf /tmp/wanderland-aws-pack

RUN gem install aws-sdk-cloudformation

WORKDIR /opt/cfn-tools
COPY config.yml /opt/cfn-tools/
COPY lib        /opt/cfn-tools/lib/

WORKDIR /workspace

Anatomy:

Step 7. The build script

The image gets built by adding one entry to vivarta.yml's pipeline. Same containerized_deployment archetype that builds the controller, agent, and cool-tools images iterates this one too.

Under pipeline.artifacts.images, add a sibling entry:

pipeline:
  artifacts:
    images:
      # ...existing controller / agent / cool-tools entries...

      cfn-tools:
        type: docker_image
        args:
          context_path: images/cfn-tools
          image_name:   !Environment cfn_tools_image_name
          image_tag:    !Reference pipeline.metadata.tags.version
          docker_build_args:
            - --build-context
            - wanderland-core=../wanderland-core
            - --build-context
            - wanderland-aws-pack=../wanderland-aws-pack

And the image name in each environment block:

environments:
  local:
    # ...existing entries...
    cfn_tools_image_name: host.docker.internal:5001/my-cfn-tools

A narrow route for incremental rebuilds (mirrors /build-cool-tools from the secrets tutorial):

routes:
  # ...existing routes...
  /build-cfn-tools:
    method: post
    name: build-cfn-tools
    chain:
      - boundary: docker_build
        args:
          source_path: images/cfn-tools
          image_name:  !Environment cfn_tools_image_name
          image_tag:   !Reference pipeline.metadata.tags.version
          docker_build_args:
            - --build-context
            - wanderland-core=../wanderland-core
            - --build-context
            - wanderland-aws-pack=../wanderland-aws-pack
      - boundary: publish_image
        args:
          image_name: !Environment cfn_tools_image_name
          image_tag:  !Reference pipeline.metadata.tags.version

And the matching rake task:

namespace "cfn-tools" do
  desc "Build and push the cfn-tools image"
  task :build do
    sh "#{W} build-cfn-tools"
  end
end

Anatomy:

Walk dry-run before committing:

bundle exec wanderland --type cli vivarta.yml build-cfn-tools --wanderland-dry-run

Step 8. The pod template

Add a third templates[] entry in jcasc/jenkins.yaml, sibling to vivarta-agent and cool-tools:

  - name: "cfn-tools"
    namespace: "vivarta-ci"
    label: "cfn-tools"
    serviceAccount: "jenkins-agent"
    yaml: |
      apiVersion: v1
      kind: Pod
      spec:
        containers:
          - name: vivarta
            image: ${CFN_TOOLS_IMAGE_REF}
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["999999"]
            tty: true
            resources:
              requests: { cpu: "200m", memory: "256Mi" }
              limits:   { cpu: "1",    memory: "1Gi" }

Plus a CFN_TOOLS_IMAGE_REF env on the controller deployment (same pattern the secrets tutorial established for COOL_TOOLS_IMAGE_REF).

Anatomy:

Step 9. The project entry

projects:
  - name: cfn_verify_workload_network
    description: Verify the advaita-nonprod-00-network CFN stack before deploy.
    git_url: git@github
    config_path: vivarta-jenkins/images/cfn-tools/config.yml
    agent_label: cfn-tools
    route: verify
    args:
      stack_name: advaita-nonprod-00-network
    dry_run: false
    environments:
      mode: single
      from: [nonprod]
    env:
      CFN_TEMPLATE: cloudformation/workload/templates/00-network.yaml
      CFN_PARAMS:   cloudformation/workload/params/nonprod/00-network.yaml
      CFN_CONFIG:   cloudformation/config.yaml
      CFN_LINT_THRESHOLD: warning
    secrets:
      - id: aws-creds-${WANDERLAND_ENVIRONMENT}
        bind: usernamePassword
        env:
          username: AWS_ACCESS_KEY_ID
          password: AWS_SECRET_ACCESS_KEY

(Drop the secrets: block if your cluster has IRSA and the agent pod's service account carries the IAM role.)

Step 10. Apply, build, verify

Two commands — build the image, apply the JCasC and deployment changes, watch the controller roll:

rake cfn-tools:build   # docker_build + publish_image for cfn-tools
rake jcasc:apply       # bake configmap + apply overlay + rollout restart

rake cfn-tools:build walks the narrow /build-cfn-tools route added in Step 7 — two docker shell calls (build, push) against the entries in vivarta.yml. rake jcasc:apply walks /apply-jcasc — bake_jcasc_configmap regenerates k8s/base/jcasc-configmap.yaml, then kubectl_apply runs apply -k, rollout restart, and rollout status. The new pod template lands when JCasC reloads.

Build the cfn_verify_workload_network job. A cfn-tools pod spawns. The pipeline body calls:

wanderland --type cli /opt/cfn-tools/config.yml verify \
  --stack-name advaita-nonprod-00-network

The wrapper runs:

A successful run logs a JSON crossing that lists every resource AWS would build, every parameter value AWS resolved, and any lint findings below threshold. Compare against what the runbook says the stack should create — discrepancy = bug somewhere, fix both ends.

Halt paths

Record and replay

cfn_verify reaches two adapters — :cloudformation (kind: aws) and :cfn_lint (kind: shell). Both carry the same four-mode dispatch as the HTTP adapter: live (real SDK / real shellout), replay (recorded responses from disk), mock (scenario-stubbed responses), dry_run (logged, never invoked). Recording once gives you a cassette set the next replay-mode run serves offline — useful for CI runs that don't have AWS credentials, useful for demoing without an account to hand, useful for catching regressions when an upstream API drifts.

Three steps: record, inspect, replay.

Record. From the cfn-tools image directory (vivarta-jenkins/images/cfn-tools/), run the route once with --wanderland-record-to:

AWS_REGION=us-west-2 \
CFN_TEMPLATE=cloudformation/workload/templates/00-network.yaml \
CFN_PARAMS=cloudformation/workload/params/nonprod/00-network.yaml \
CFN_CONFIG=cloudformation/config.yaml \
CFN_LINT_THRESHOLD=warning \
bundle exec wanderland --type cli config.yml verify \
  --stack-name advaita-nonprod-00-network \
  --wanderland-record-to spec/fixtures/cassettes

render_params is a pure transform — no adapter, no cassette. cfn_validate_template, cfn_change_set_preview go through the :cloudformation adapter (kind: aws): each SDK call writes a cassette under spec/fixtures/cassettes/cloudformation/. cfn_lint_template goes through the :cfn_lint adapter (kind: shell): the cfn-lint invocation writes a cassette under spec/fixtures/cassettes/cfn-lint/.

Inspect. A representative AWS cassette:

ls spec/fixtures/cassettes/cloudformation/
# 4f1a09cb...yml  # validate_template
# 9c8e1d44...yml  # create_change_set
# a72b4519...yml  # describe_change_set
# d139b502...yml  # delete_change_set

cat spec/fixtures/cassettes/cloudformation/4f1a09cb....yml
---
request:
  operation: validate_template
  params:
    template_body: |
      AWSTemplateFormatVersion: '2010-09-09'
      Description: workload/00-network
      ...
response:
  data:
    parameters: []
    capabilities: []
    capabilities_reason:

And the cfn-lint cassette:

cat spec/fixtures/cassettes/cfn-lint/bb50....yml
---
request:
  args: ["--format", "json", "cloudformation/workload/templates/00-network.yaml"]
  cwd: /workspace
response:
  exit_code: 0
  stdout: "[]\n"
  stderr: ""
  duration_s: 0.42

The shell cassette also keys on cwd and an env subset — re-running with the same args but a different working directory writes a new cassette. The AWS cassette keys on the SDK operation name and the request params hash; same operation with the same params replays the same response.

Replay. Flip both adapters' modes at invocation:

WANDERLAND_AWS_MODE=replay \
WANDERLAND_SHELL_MODE=replay \
CFN_TEMPLATE=cloudformation/workload/templates/00-network.yaml \
CFN_PARAMS=cloudformation/workload/params/nonprod/00-network.yaml \
CFN_CONFIG=cloudformation/config.yaml \
CFN_LINT_THRESHOLD=warning \
bundle exec wanderland --type cli config.yml verify \
  --stack-name advaita-nonprod-00-network

Same input, no network, no shellout. The adapters compute cassette keys from the planned requests, read the YAML, return the recorded responses. A request the cassettes don't cover raises CassetteMissError with the file path the adapter looked for — drift fails loudly instead of silently succeeding.

!Env { name: WANDERLAND_AWS_MODE, default: live } (Step 5's config.yml) is the resolver wired into the adapter's mode:. The env flips at invocation; no config edit needed.

Re-recording one interaction. Delete the cassette for the call that drifted and re-run with --wanderland-record-to while back on mode: live. The one missing interaction lands; the rest stay frozen. A git diff against the cassette directory shows exactly what changed upstream.

Mock for scenario-specific shapes. Replay covers "what does this boundary do against the real AWS." For "what does it do when the API returns a shape you can't reliably get on demand" — a throttling error, an IAM denial, a FAILED change set — declare a scenario in config.yml:

scenarios:
  verify:
    iam_denied:
      description: "CreateChangeSet returns AccessDenied — boundary halts cleanly"
      input:
        params: { stack_name: advaita-nonprod-00-network }
      expected:
        status: 422
        halted_at: cfn_change_set_preview
      mocks:
        cloudformation:
          - !Fixture cassettes/cloudformation/validate_template_ok
          - request: { operation: create_change_set }
            response:
              error:
                class: Aws::CloudFormation::Errors::AccessDenied
                message: "User: arn:aws:iam::...:role/cfn-tools is not authorized to perform cloudformation:CreateChangeSet"
        cfn-lint:
          - !Fixture cassettes/cfn-lint/clean_run

Run with WANDERLAND_AWS_MODE=mock WANDERLAND_SHELL_MODE=mock bundle exec wanderland --type test config.yml --scenario iam_denied and the wrapper walks against canned responses — the first two steps from cassettes lifted as fixtures, the third stubbed inline to surface the IAM error.

See Adapter Modes and AWS Adapter Modes for the deeper walks — cassette digest scheme, !Fixture lifting, !Merge and !Patch for twisting one field of a recorded response, the #client escape hatch for SDK features the uniform facade doesn't expose.

What you have now

A single cfn_verify boundary on a route. Users invoke it with one call and get a deterministic answer to "would my template + params deploy correctly." The four sub-boundaries are reusable — cfn_change_set_preview will be the same code when we later build a cfn_stack_deploy wrapper that swaps the preview step for a real create + wait.

Three modes work the same way:

What's missing

See also