Kubernetes Sandbox

K8sEphemeralProvider::sandboxed(image) runs every agent step in a hardened, single-use pod. Everything is opt-in: K8sEphemeralProvider::new(image) keeps its behaviour, apart from the run/step labels, the expiry annotation and the cleanup of a previous attempt described below.

The image

The official image registry.gitlab.com/thomastartrau/ironflow/ironflow-claude-runner:<claude-code-version>-<n> is built from docker/claude-runner/ by the build-claude-runner-image CI job. The current tag is in docker/claude-runner/IMAGE_TAG. There is no latest tag: pin the full tag. The image also bundles chrome-devtools-mcp at a pinned version, with no browser, for agents that drive a remote Chromium.

# Official ironflow Claude Code runner image, used by
# K8sEphemeralProvider::sandboxed.
#
# Published as ironflow-claude-runner:<claude-code-version>-<n>, the content of
# IMAGE_TAG next to this file. Never `latest`. CLAUDE_CODE_VERSION is the
# <claude-code-version> part: see README.md for a local build.
#
# Layout:
#   /home/claude                 HOME of uid 10001. The sandboxed provider mounts
#                                an emptyDir here: the root filesystem is read-only.
#   /tmp                         emptyDir mounted by the sandboxed provider.
#   /etc/claude-code             managed-settings.json, read by Claude Code on
#                                Linux. A default is baked in; a managed-settings
#                                ConfigMap mounted by the provider replaces it.
#   /etc/ironflow/claude-profile Claude profile ConfigMap (CLAUDE.md, settings,
#                                agents), copied into ~/.claude before the agent
#                                starts because ~/.claude must stay writable.
#
# Both /etc directories are owned by root with mode 0755: the agent can read
# them, never change them.

# Pinned by digest so a rebuild of the same tag starts from the same base. To
# move to a newer node image, replace the digest and bump <n> in IMAGE_TAG.
# 22.23.4 fixes Node2Shell (CVE-2026-111497, Brotli decoder).
FROM node:22.23.4-bookworm-slim@sha256:de170b39439eb0e5a3444ad7d60b015614a381e10fecae2e30ec9e735122f38d

ARG CLAUDE_CODE_VERSION

# upgrade: the base image ships libbrotli1 1.0.9-2+b6 (CVE-2026-111497, decoded by
# libcurl under git and curl), fixed in 1.0.9-2+deb12u1.
RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install -y --no-install-recommends ca-certificates git curl \
    && rm -rf /var/lib/apt/lists/*

# Fails the build when the version is missing, or when the installed CLI
# reports another one.
RUN : "${CLAUDE_CODE_VERSION:?pass --build-arg CLAUDE_CODE_VERSION=<version>}" \
    && npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" \
    && npm cache clean --force \
    && test "$(claude --version | cut -d' ' -f1)" = "${CLAUDE_CODE_VERSION}"

# chrome-devtools-mcp: stdio MCP server the Claude CLI starts in the agent pod,
# driving a browser that runs in another pod (--browser-url
# http://<pod-ip>:9223). No Chromium here. Installed at an exact version because
# the agent pod has no egress to the npm registry, so `npx` at startup cannot work
# and a floating download is not reproducible. Fails the build when the
# installed server reports another version.
RUN PUPPETEER_SKIP_DOWNLOAD=1 npm install -g chrome-devtools-mcp@1.10.1 \
    && npm cache clean --force \
    && test "$(chrome-devtools-mcp --version)" = "1.10.1"

RUN groupadd -g 10001 claude \
    && useradd -u 10001 -g 10001 -m -d /home/claude claude

# Marketplace neutralisation: no plugin marketplace, no hooks, no auto-update,
# no non-essential traffic. A mounted managed-settings ConfigMap replaces the
# baked file.
RUN mkdir -p /etc/claude-code /etc/ironflow/claude-profile \
    && printf '%s\n' '{"strictKnownMarketplaces": [], "disableAllHooks": true}' \
        > /etc/claude-code/managed-settings.json \
    && chown -R root:root /etc/claude-code /etc/ironflow \
    && chmod 0755 /etc/claude-code /etc/ironflow /etc/ironflow/claude-profile \
    && chmod 0644 /etc/claude-code/managed-settings.json

ENV CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 \
    DISABLE_AUTOUPDATER=1 \
    HOME=/home/claude

USER 10001:10001
WORKDIR /home/claude

sandboxed() defaults

A relaxation called on a provider built with new() panics.

Secrets

A sandboxed provider refuses oauth_credentials(json) and env("ANTHROPIC_API_KEY", ..) / env("CLAUDE_CODE_OAUTH_TOKEN", ..): the value would sit in clear text in the pod spec. Read it from a Kubernetes Secret instead; the pod only carries a secretKeyRef:

let provider = K8sEphemeralProvider::sandboxed(&image)
    // Long-lived token from `claude setup-token`.
    .oauth_token_from_secret("claude-oauth", "token")
    // Or the full credentials JSON, written to ~/.claude/.credentials.json.
    .oauth_credentials_from_secret("claude-credentials", "credentials.json")
    // Any other variable.
    .env_from_secret("GITLAB_TOKEN", "gitlab-bot", "token");

Auth proxy: no Claude credential in the pod

A secretKeyRef keeps the credential out of the pod spec, not out of the pod: anything running in the agent container can still read CLAUDE_CODE_OAUTH_TOKEN. With auth_proxy, the pod never receives it:

let provider = K8sEphemeralProvider::sandboxed(&image)
    .namespace("ironflow-agents")
    .auth_proxy("http://ironflow-auth-proxy.ironflow-system");

The ironflow-auth-proxy service (crate ironflow-auth-proxy, manifests in examples/k8s/sandbox/auth-proxy.yaml) holds the credential instead. Its official image registry.gitlab.com/thomastartrau/ironflow/ironflow-auth-proxy:<version> is built from docker/auth-proxy/Dockerfile by the build-auth-proxy-image CI job. <version> is the version of the ironflow-auth-proxy crate: the job publishes it once that version is released, and never rebuilds a published tag. There is no latest tag: pin the version.

• Token lifecycle. At pod launch the worker calls the proxy admin API and gets an opaque token (ifap_...) bound to the run id, the step and the pod expiry (ironflow.io/expires-at). The pod receives ANTHROPIC_BASE_URL (the proxy URL), ANTHROPIC_AUTH_TOKEN (the opaque token) and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1. The worker revokes the token when the step ends, whatever the outcome, including a step dropped before it ends (run cancelled, lease lost, run or workflow timed out, parallel branch aborted), and every token of a run when the run is released; the expiry is the backstop, after a worker crash or shutdown. A step that ends waits at most 5 seconds for the revocation, which goes on in the background past it. A cancelled run is dropped at the next lease renewal (every 30 s by default). An unknown, revoked or expired token gets a 401. If the proxy cannot issue a token, the step fails before any pod is created.
• Restrictions. The proxy only relays GET and POST requests under /v1/ to api.anthropic.com. A path outside /v1/ (including /admin, which needs the admin key) or a request for another host (absolute-form URI, CONNECT) gets a 403, any other method a 405.
• Which credential. The step’s Provider Account when one is attached (the provider reports the claude_subscription account kind once auth_proxy is set), else CLAUDE_CODE_OAUTH_TOKEN, then ANTHROPIC_API_KEY, from the worker environment. The worker also needs IRONFLOW_AUTH_PROXY_ADMIN_KEY (or .auth_proxy_admin_key(..)). Rate-limit windows reported through the proxy are recorded on the Provider Account as with the Docker provider.
• No credential on the pod side. With auth_proxy set, a Claude credential configured for the pod (oauth_token_from_secret, oauth_credentials, oauth_credentials_from_secret, a step env_from_secret("CLAUDE_CODE_OAUTH_TOKEN", ..), any plain value starting with sk-ant) fails the step with an error naming the variable.
• Registry. In memory by default (one replica). Set IRONFLOW_AUTH_PROXY_DATABASE_URL and an encryption key (IRONFLOW_SECRET_KEYS) for a shared PostgreSQL registry: several replicas, tokens survive restarts, the proxy needs egress to PostgreSQL. Under Cilium, uncomment the 5432 rule in section (b) of cilium-egress-auth-proxy.yaml (below). With standard NetworkPolicies only, nothing restricts the proxy’s egress unless the operator adds a policy on the proxy pods.
• Logs. The proxy and the worker log the first 12 characters of the token id (a SHA-256 of the token), never the token or the credential.

The network side changes too: the agent pods only reach the proxy, and the proxy only reaches api.anthropic.com (and PostgreSQL with the shared registry):

# Cilium egress for the auth proxy variant.
#
# Use INSTEAD of cilium-egress-anthropic.yaml: the agent pods no longer reach
# api.anthropic.com, only the auth proxy, which alone holds the credential
# and alone reaches the API. The per-profile policies (gitlab below) are
# unchanged and stay additive. With proxied secrets (the GitLab token held by
# the proxy, git and the API reached through /r/), the agent pods no longer
# need the gitlab profile: open gitlab.com on the proxy pods instead, in (b).
#
# (a) Agent pods: DNS, and the auth proxy pods on 8080.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: claude-runner-egress-auth-proxy
  namespace: ironflow-agents
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/component: claude-runner
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s:k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchPattern: "*"
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: ironflow-system
            app.kubernetes.io/name: ironflow-auth-proxy
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
---
# Per-profile opening, as in cilium-egress-anthropic.yaml: pods labelled
# ironflow.io/egress-profile=gitlab may also reach gitlab.com.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: claude-runner-egress-gitlab
  namespace: ironflow-agents
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/component: claude-runner
      ironflow.io/egress-profile: gitlab
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s:k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchPattern: "*"
    - toFQDNs:
        - matchName: gitlab.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
---
# (b) The auth proxy pods: egress to DNS and api.anthropic.com:443 only, plus
# the hosts of the proxied secrets (commented-out rule below), plus
# PostgreSQL on 5432 with the shared registry (commented-out rule below);
# ingress on 8080 only from the agent pods and from the worker. Adjust the
# worker selector to the namespace and labels the worker runs under.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: ironflow-auth-proxy
  namespace: ironflow-system
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: ironflow-auth-proxy
  ingress:
    - fromEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: ironflow-agents
            app.kubernetes.io/component: claude-runner
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: ironflow
            app.kubernetes.io/name: ironflow-worker
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s:k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchPattern: "*"
    - toFQDNs:
        - matchName: api.anthropic.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
    # Proxied secrets: every host in the allowlist of a proxied secret must be
    # opened here for the proxy pods, else the relay answers 502. Wildcard
    # allowlist entries map to matchPattern.
    # - toFQDNs:
    #     - matchName: api.github.com
    #     - matchName: gitlab.com
    #   toPorts:
    #     - ports:
    #         - port: "443"
    #           protocol: TCP
    #
    # Shared PostgreSQL registry (IRONFLOW_AUTH_PROXY_DATABASE_URL set in
    # auth-proxy.yaml): uncomment ONE of the two rules below, else the proxy
    # times out at startup. Not needed with the in-memory registry. If the
    # database has its own ingress policy, open 5432 there to the proxy pods.
    #
    # In-cluster database: adjust its namespace and pod labels.
    # - toEndpoints:
    #     - matchLabels:
    #         k8s:io.kubernetes.pod.namespace: ironflow-db
    #         app.kubernetes.io/name: postgresql
    #   toPorts:
    #     - ports:
    #         - port: "5432"
    #           protocol: TCP
    #
    # Managed database outside the cluster: adjust to its subnet.
    # - toCIDR:
    #     - 10.0.0.0/24
    #   toPorts:
    #     - ports:
    #         - port: "5432"
    #           protocol: TCP

Proxied secrets

The same proxy keeps other secrets out of the pod: a GitHub token, a GitLab token, any API key. proxied_secret declares one on the provider (every step) or on a step, a step entry replacing a provider entry with the same env:

use std::env::var;

use ironflow_core::auth_proxy::{ProxiedSecret, SecretInjection};

let provider = K8sEphemeralProvider::sandboxed(&image)
    .auth_proxy("http://ironflow-auth-proxy.ironflow-system")
    .proxied_secret(ProxiedSecret {
        name: "GITHUB_TOKEN".to_string(),
        env: "GITHUB_TOKEN".to_string(),
        value: var("GITHUB_TOKEN")?,
        injection: SecretInjection::Bearer,
        hosts: vec!["api.github.com".to_string()],
    });

let config = AgentConfig::new("Open the merge request").proxied_secret(ProxiedSecret {
    name: "GITLAB_TOKEN".to_string(),
    env: "GITLAB_TOKEN".to_string(),
    value: var("GITLAB_TOKEN")?,
    injection: SecretInjection::Basic { username: "oauth2".to_string() },
    hosts: vec!["gitlab.com".to_string()],
});

The worker issues one opaque token per secret, bound to the run and the step like the Claude token and revoked with it. The pod receives the token in <env> and the relay base <proxy>/r in <env>_URL, never the value, and calls $<env>_URL/<host>/<path> presenting the token as Authorization: Bearer, x-api-key, Private-Token or the password of Authorization: Basic. The proxy strips it and injects the real secret:

• Allowlist. hosts holds exact host names or a leading wildcard (*.example.com matches api.example.com, not example.com). No regex, no port, no IP address; the proxy only speaks https upstream. A host outside the list, a path with .., // or percent-encoding, a Claude token on /r/ and a secret token on the Anthropic API get a 403; an unknown, expired or revoked token a 401. The query string is relayed untouched.
• git. A Basic secret also sets the git config of the pod through GIT_CONFIG_COUNT / GIT_CONFIG_KEY_n / GIT_CONFIG_VALUE_n: for each exact host, https://<host>/ is rewritten to <proxy>/r/<host>/, so git clone https://gitlab.com/group/repo.git works unchanged. A step setting a GIT_CONFIG_* variable itself is refused, as is a proxied <env> or <env>_URL the pod already receives.
• Redirects are returned to the pod with their Location, never followed: the secret never travels to another host.
• Logs and metrics. Each /r/ request logs host, secret (the name), run_id, step, the short token id, result and upstream_status, never the token, the secret, a header or the query string. The counter ironflow_auth_proxy_requests_total{secret, result} is served at /metrics on the proxy port.
• Egress. The proxy pods need egress to every allowlisted host: uncomment the proxied secrets toFQDNs rule in section (b) of cilium-egress-auth-proxy.yaml. The agent pods then no longer need the gitlab egress profile.

Per-step settings

A step adds to or overrides the provider’s settings through its AgentConfig. Other providers ignore these fields.

let config = AgentConfig::new("Open the merge request")
    .env_from_secret("GITLAB_TOKEN", "gitlab-bot", "token") // wins over the provider's
    .service_account("gitlab-reader")                       // wins over the provider's
    .read_only_pvc("repos", "/data/repos")                  // after the provider's volumes
    .read_only_config_map("guidelines", "/data/guidelines")
    .managed_settings("readonly")                           // preset registered on the provider
    .runtime_class("gvisor")                                // wins over the provider's
    .egress_profile("gitlab");                              // ironflow.io/egress-profile label

Read-only mounts must be absolute, unique, and cannot shadow /, /home/claude, /tmp, /etc/claude-code, or /etc/ironflow/claude-profile and anything below it.

Claude profile

A Claude profile (CLAUDE.md, settings.json, rules/, agents/, commands/) reaches ~/.claude through ConfigMaps. A ConfigMap key cannot contain /, so each directory of the profile is its own ConfigMap, mapped to its sub-directory:

let provider = K8sEphemeralProvider::sandboxed(&image)
    .claude_profile_configmap("claude-profile")                 // ~/.claude
    .claude_profile_configmap_at("claude-profile-rules", "rules") // ~/.claude/rules
    .claude_profile_configmap_at("claude-profile-agents", "agents");

Each ConfigMap is mounted read-only in its own directory, then only its keys are copied into ~/.claude/<subdir> before the agent starts: never the ..data entries of the volume. Profiles are copied before the credentials, which they cannot overwrite. A subdir must be relative, made of [A-Za-z0-9._-] segments without . or .., and mapped once: the provider panics at build time otherwise.

With kustomize, one configMapGenerator per directory. Disable the name hash: the provider refers to the ConfigMaps by name.

# kustomization.yaml, next to claude-home/
namespace: ironflow-agents
generatorOptions:
  disableNameSuffixHash: true
configMapGenerator:
  - name: claude-profile
    files:
      - claude-home/CLAUDE.md
      - claude-home/settings.json
  - name: claude-profile-rules
    files:
      - claude-home/rules/rust.md
      - claude-home/rules/security.md

kustomize lists every file. To pick up a whole directory instead: kubectl create configmap claude-profile-rules --from-file=claude-home/rules/ --dry-run=client -o yaml.

Managed-settings presets map a name to a ConfigMap holding managed-settings.json. An unknown preset fails the step, never falls back:

let provider = K8sEphemeralProvider::sandboxed(&image)
    .managed_settings_preset("locked", "claude-managed-locked")
    .managed_settings_preset("readonly", "claude-managed-readonly")
    .default_managed_settings("locked");

Labels and retry cleanup

The engine stamps ironflow.io/run-id, ironflow.io/root-run-id and ironflow.io/step on every agent step (outside the engine, call AgentConfig::run_scope(run_id, step)). Step names are sanitized into valid label values, with a hash suffix when altered. The root run is the run itself, or the top-level run inside a sub-workflow.

Before every execution of a run, first one included, the engine calls release_run: the provider deletes every pod, JobRun Job and prompt ConfigMap labelled with the run id or with the run as root, and waits until the pods are gone (.previous_attempt_timeout(d), 60s by default). A retry that starts by resetting shared state (a worktree) never runs next to an agent of the dead attempt still writing to it. If the pods are still there, or the Kubernetes API fails, the execution fails with a replayable error and the next attempt tries again.

Tag a pod you create yourself with the same labels so that it is released too:

let run = PodRun::new(&kube, "check", &image, "cargo test")
    .label(LABEL_RUN_ID, &ctx.run_id().to_string())
    .label(LABEL_ROOT_RUN_ID, &ctx.root_run_id().to_string());

A service pod started with PodUp::new(run, lifetime) returns once ready and keeps running; tag it with the labels above on its PodRun, and its ironflow.io/expires-at covers lifetime. Only a pod in the provider’s namespace is released or reaped: elsewhere, put the run id in its name, or a new attempt of the run finds the old pod in the way. Once tagged, it is released when the run executes again, while its step is replayed with the IP of the deleted pod: keep the steps that use the service in the same execution, with no approval, signal or human input between them and the PodUp.

Before creating a pod, the provider also deletes the pods and prompt ConfigMaps of a previous attempt of the same step of the same run. If they are still terminating, the step fails: two agents never run side by side. Two branches of a ctx.parallel() group with the same name would delete each other’s pod, so the engine fails such a group before creating any step.

Deleting JobRun Jobs needs list and delete on jobs; without them, Jobs are skipped with a warning and the pods are still released.

Tying pods to the worker pod

release_run only runs when the run executes again, after the worker lease expired. Until then, the agent pod of a killed worker keeps running, and writing to shared state. .owner_pod(name, uid) puts an ownerReference to the worker pod on every agent pod and prompt ConfigMap (controller: false, blockOwnerDeletion: false): the Kubernetes garbage collector deletes them as soon as the worker pod is gone. PodRun::owner_pod and JobRun::owner_pod do the same for the pods and Jobs of ironflow-ops-k8s; a Job takes its pods with it.

The worker reads its name and uid through the downward API:

env:
  - name: POD_NAME
    valueFrom: { fieldRef: { fieldPath: metadata.name } }
  - name: POD_UID
    valueFrom: { fieldRef: { fieldPath: metadata.uid } }

let (name, uid) = (var("POD_NAME")?, var("POD_UID")?);
let provider = K8sEphemeralProvider::sandboxed(&image)
    .namespace(&namespace)
    .owner_pod(&name, &uid);
// A container restarted in the same pod (OOM, crash) keeps its pod, so the
// garbage collector leaves its agents running: release them at boot.
provider.release_owned().await?;
let check = PodRun::new(&kube, "check", &image, "cargo test")
    .namespace(&namespace)
    .owner_pod(&name, &uid);

An owner reference only holds in the namespace of its owner: with an owner the garbage collector cannot find in the namespace of the pod, it deletes the pod right after its creation. The worker must therefore run in the namespace of its agent pods, unlike the two-namespace layout of examples/k8s/sandbox/namespace-rbac.yaml. Before creating anything, each invocation, PodRun::run and JobRun::run read the owner pod there and fail when it is missing or carries another uid. An empty name or uid fails too, before any request.

release_owned deletes the ironflow pods, JobRun Jobs (and their pods) and prompt ConfigMaps with an ownerReference to the owner’s uid, and waits until the pods are gone (.previous_attempt_timeout(d)). Objects of another worker or without owner are left alone. It fails when no owner is set.

PodRun and JobRun keep at most 1 MiB of logs: past that, only the tail is kept, behind a [... N bytes truncated ...] marker, and logs_truncated is set on the output. .max_log_bytes(n) changes the limit. A PodRun with .security(..) sets fsGroupChangePolicy: OnRootMismatch, so a large volume is not re-chowned on every pod; .fs_group_change_policy("Always") restores the Kubernetes default.

A PodRun running a script it does not trust takes .sandboxed(): the same hardening as K8sEphemeralProvider::sandboxed (read-only root, all capabilities dropped, seccomp RuntimeDefault, non-root uid 10001 unless .security(..) sets one, /tmp on a 512Mi emptyDir, no service account token). .stdin(input) feeds the script up to 255 KiB through a downward API file, and the output reports how it ended: exit_code from the container, reason from the pod (DeadlineExceeded when active_deadline_seconds killed it, with no exit_code) or from the container (OOMKilled, Error). logs holds stdout and stderr interleaved; redirect stderr in the command to parse stdout alone.

let out = ctx
    .operation(
        "extract",
        &PodRun::new(&kube, "extract", &image, "python /scripts/extract.py 2>/dev/null")
            .sandboxed()
            .stdin(document_json)
            .active_deadline_seconds(Duration::from_secs(60)),
    )
    .await?;
let timed_out = out.reason.as_deref() == Some("DeadlineExceeded");

A read-only root needs writable volumes: .scratch_volume(name, mount_path, size, storage_class) adds one. Without a storage class it is an emptyDir bounded by size; with one it is an ephemeral claim created and deleted with the pod. Tolerations written as key[=value]:Effect (comma-separated, key alone means Exists) are parsed once by K8sToleration::parse_list and apply to the agent providers and to a PodRun alike, through .k8s_tolerations(..).

let tolerations = K8sToleration::parse_list("dedicated=worker:NoSchedule,gpu:NoExecute")?;
let provider = tolerations
    .iter()
    .cloned()
    .fold(K8sEphemeralProvider::sandboxed(&image), |p, t| p.toleration(t));
let run = PodRun::new(&kube, "build", &image, "make")
    .sandboxed()
    .k8s_tolerations(tolerations)
    .scratch_volume("target", "/target", "10Gi", None)
    .scratch_volume("cache", "/cache", "50Gi", Some("fast-ssd"));

Persistent environment

By default an agent pod starts from an empty working directory and loses everything when it exits. environment_volume gives each agent step a ReadWriteOnce PersistentVolumeClaim that outlives the pod, so a later step can continue in the files an earlier one left (a clone, a build cache, a half-written patch):

use ironflow_core::provider::{EnvironmentVolume, StorageUnit, VolumeSize};

let provider = K8sEphemeralProvider::sandboxed(&image)
    .environment_volume(
        EnvironmentVolume::new("/workspace")
            .size(VolumeSize::new(20, StorageUnit::Gi)) // 10Gi by default
            .storage_class("fast-ssd")           // cluster default otherwise
            .ttl(Duration::from_secs(3 * 86400)), // 7 days by default
    )
    .working_dir("/workspace");

A step without a resume id gets a new claim named ironflow-env-...; its name comes back as the environment id, in StepOutput::environment_id (and AgentResult::environment_id outside the engine). The id is stored with the step, shown by the API and replayed with the step when a run resumes. Pass it to the next step to mount the same claim again:

let clone = ctx
    .agent(
        "clone",
        AgentStepConfig::new("Clone the repository and run the tests").allow_tool(Tool::Bash),
    )
    .await?;
// `None` on a provider without persistent environments.
if let Some(environment) = clone.environment_id.as_deref() {
    ctx.agent(
        "fix",
        AgentStepConfig::new("Fix the failing test")
            .allow_tool(Tool::Bash)
            .resume_environment(environment),
    )
    .await?;
}

A structured step (.output::<T>()) answers with T, which hides the id from ctx.agent. Use ctx.agent_with_meta: its AgentReply holds the typed answer, the environment_id and the account_id. A replayed step returns the same ids.

The claim carries the ironflow labels with app.kubernetes.io/component=environment, the run and step labels of the step that created it, and the ironflow.io/expires-at annotation. Every use pushes the expiry ttl forward (never less than the pod lifetime). A resumed step fails before any pod is created when the claim does not exist, is not an ironflow environment, or is being deleted. A step that sets resume_environment on a provider without environment_volume fails the same way. Other providers return no environment id and ignore the resume id.

The claim is ReadWriteOnce: two steps resuming the same environment at the same time can only run on the same node. Run them one after the other. There is no snapshot: a step that breaks the workspace leaves it broken for the next one.

The worker needs create, get, patch, list and delete on persistentvolumeclaims (see examples/k8s/sandbox/namespace-rbac.yaml).

Sessions volume

An agent step interrupted by a lost lease resumes its Claude Code session on its next execution (see Resuming an interrupted agent step). HOME of a sandboxed pod is an emptyDir, so the session dies with the pod and the step restarts from scratch. sessions_volume mounts an existing PersistentVolumeClaim on ~/.claude/projects, where Claude Code writes its sessions, so the next pod finds the session again:

let provider = K8sEphemeralProvider::sandboxed(&image)
    .sessions_volume("claude-sessions")
    .working_dir("/workspace");

The claim is yours to create; the provider never creates nor deletes it. Use a ReadWriteMany claim, or make sure the next pod lands on the same node. Claude Code keys its sessions by working directory: keep the same working_dir across executions. A Claude profile mapped to projects (or below) and a step volume mounted on ~/.claude/projects are refused, since they would hide the sessions.

The reaper

Every object ironflow creates carries app.kubernetes.io/managed-by=ironflow, an app.kubernetes.io/component (claude-runner, prompt-data, pod-run, job-run) and the ironflow.io/expires-at annotation (unix seconds: creation

• timeout + margin). That covers the agent pods and their prompt ConfigMaps, and the pods of PodRun and Jobs of JobRun from ironflow-ops-k8s (.expiry_margin(d), 60s by default). A caller cannot set managed-by or component: pod_label, PodRun::label and JobRun::label panic, an agent step carrying one fails.

The reaper selects on managed-by=ironflow and deletes pods and Jobs past their expiry or killed with DeadlineExceeded, expired prompt ConfigMaps and expired environment claims (a claim being deleted is left alone). A Job goes with its pods; a pod a Job controls is left to its Job. Objects without a parseable annotation are never touched.

let report = provider.reap_orphans().await?;            // one pass
let handle = provider.spawn_orphan_reaper(Duration::from_secs(300)); // background
// Without a K8sEphemeralProvider, e.g. a worker that only runs PodRun:
let report = reap_orphans(&K8sClusterConfig::Default, "ironflow-agents").await?;

Reaping Jobs needs list and delete on jobs (see examples/k8s/sandbox/namespace-rbac.yaml). Without them, the Job pass is skipped with a warning and the rest of the pass runs. The same holds for list and delete on persistentvolumeclaims and the environment pass.

gVisor (RuntimeClass)

A runc container shares the node kernel: a kernel exploit in untrusted code reaches the node. gVisor runs the pod against a user-space kernel instead. On Talos, install the gvisor system extension on the nodes, then declare the RuntimeClass once per cluster:

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc

Set it on the provider to cover every agent pod, and override it for one step (the step wins):

let provider = K8sEphemeralProvider::sandboxed(&image)
    .runtime_class("gvisor");

let config = AgentConfig::new("Review this untrusted patch")
    .runtime_class("kata"); // wins over the provider's

A check pod created with PodRun takes the same setting:

let run = PodRun::new(&kube, "check", "rust:1.94", "cargo test")
    .runtime_class("gvisor");

Without a value, spec.runtimeClassName stays absent and the cluster default runtime (runc) applies. A blank name is refused.

Warning: builds are noticeably slower under gVisor (compilation, many small file syscalls, cargo and npm installs). Keep gVisor for the steps that handle untrusted code and leave trusted build steps on runc.

Network policies

The worker does not create network policies: its Role in examples/k8s/sandbox/namespace-rbac.yaml has no access to them. A cluster administrator applies a default deny (networkpolicy-deny-all.yaml) and FQDN openings selected on the ironflow.io/egress-profile label:

# FQDN egress for agent pods with Cilium.
#
# Every claude-runner pod may resolve names through kube-dns and reach
# api.anthropic.com on 443. The DNS rule with `matchPattern: "*"` is required:
# Cilium learns the IPs behind toFQDNs names by inspecting DNS answers.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: claude-runner-egress-anthropic
  namespace: ironflow-agents
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/component: claude-runner
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s:k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchPattern: "*"
    - toFQDNs:
        - matchName: api.anthropic.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
---
# Per-profile opening: pods labelled ironflow.io/egress-profile=gitlab (set
# with `.egress_profile("gitlab")` on the provider or the step) may also reach
# gitlab.com. Cilium policies are additive: this adds to the rule above.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: claude-runner-egress-gitlab
  namespace: ironflow-agents
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/component: claude-runner
      ironflow.io/egress-profile: gitlab
  egress:
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s:k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchPattern: "*"
    - toFQDNs:
        - matchName: gitlab.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP

Full example

//! Sandboxed K8s ephemeral transport example.
//!
//! Runs Claude Code in a hardened pod: non-root, read-only root filesystem,
//! all capabilities dropped, OAuth token read from a Kubernetes Secret, a
//! locked-down managed-settings preset, and the `anthropic-only` egress
//! profile label. A background task reaps orphaned pods every five minutes.
//!
//! Apply the manifests of `examples/k8s/sandbox/` first. Requires a reachable
//! Kubernetes cluster (via kubeconfig or in-cluster).
//!
//! # Usage
//!
//! ```sh
//! K8S_IMAGE=registry.gitlab.com/thomastartrau/ironflow/ironflow-claude-runner:2.1.316-1 \
//!     cargo run --bin k8s-sandboxed
//! ```

use std::env;
use std::time::Duration;

use ironflow_core::prelude::*;
use ironflow_core::providers::claude::K8sEphemeralProvider;

#[tokio::main]
async fn main() -> Result<(), OperationError> {
    let image = env::var("K8S_IMAGE").expect("K8S_IMAGE env var required");

    let provider = K8sEphemeralProvider::sandboxed(&image)
        .namespace("ironflow-agents")
        .oauth_token_from_secret("claude-oauth", "token")
        .managed_settings_preset("locked", "claude-managed-locked")
        .default_managed_settings("locked")
        .egress_profile("anthropic-only")
        .timeout(Duration::from_secs(600));
    let _reaper = provider.spawn_orphan_reaper(Duration::from_secs(300));

    // Outside the engine, `run_scope` sets the run/step labels the engine
    // stamps on every agent step.
    let config = AgentConfig::new("List the top-level directories of /data/repos.")
        .max_budget_usd(0.10)
        .read_only_pvc("repos", "/data/repos")
        .run_scope("demo-run", "investigate");

    let result = Agent::from_config(config).run(&provider).await?;

    println!("Response: {}", result.text());
    println!("Model: {}", result.model().unwrap_or("unknown"));
    println!("Duration: {}ms", result.duration_ms());

    Ok(())
}

Checks

Run these against a live agent pod before trusting the setup:

# Non-root, read-only root filesystem, capabilities dropped.
kubectl -n ironflow-agents get pod -l app.kubernetes.io/component=claude-runner \
  -o jsonpath='{.items[0].spec.containers[0].securityContext}'
# No secret value in the spec, only secretKeyRef.
kubectl -n ironflow-agents get pod <pod> -o yaml | grep -A3 CLAUDE_CODE_OAUTH_TOKEN
# No service account token mounted.
kubectl -n ironflow-agents exec <pod> -- ls /var/run/secrets/kubernetes.io/serviceaccount
# Root filesystem is read-only.
kubectl -n ironflow-agents exec <pod> -- touch /usr/local/probe
# Egress is limited to the profile.
kubectl -n ironflow-agents exec <pod> -- curl -sS -m 5 https://example.com
# The worker cannot touch network policies.
kubectl auth can-i create networkpolicies -n ironflow-agents \
  --as=system:serviceaccount:ironflow:ironflow-worker
# Run, root run and step labels, and the expiry annotation.
kubectl -n ironflow-agents get pods \
  -L ironflow.io/run-id,ironflow.io/root-run-id,ironflow.io/step \
  -o custom-columns='NAME:.metadata.name,EXPIRES:.metadata.annotations.ironflow\.io/expires-at'

The exec checks against the service account, the root filesystem and egress must fail; kubectl auth can-i must answer no.

With the auth proxy, also check that the pod holds no Claude secret and that the proxy refuses an unknown token:

kubectl -n ironflow-agents exec <pod> -- env | grep -c 'sk-ant'   # 0
kubectl -n ironflow-agents exec <pod> -- sh -c \
  'curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer invalide" "$ANTHROPIC_BASE_URL/v1/messages"'   # 401

With proxied secrets, the pod holds the opaque token only, and a host outside the allowlist is refused:

kubectl -n ironflow-agents exec <pod> -- sh -c 'echo "$GITHUB_TOKEN" | cut -c1-5'   # ifap_
kubectl -n ironflow-agents exec <pod> -- sh -c \
  'curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $GITHUB_TOKEN" "$GITHUB_TOKEN_URL/example.com/"'   # 403

The relay and the metric are covered by the proxy test suites:

cargo test -p ironflow-auth-proxy --test relay_secrets
cargo test -p ironflow-auth-proxy --test metrics