OpenKruise Agents v0.6.0: Leaping ahead in agent Sandbox security and automation

About OpenKruise Agents

OpenKruise Agents is a sub-project of OpenKruise, the CNCF open-source application automation and workload orchestration project. It provides full lifecycle management for Agent Sandboxes running at scale on Kubernetes, together with a complete runtime environment covering networking, storage, and identity. OpenKruise Agents exposes both Kubernetes and E2B interfaces, so infrastructure teams, platform teams, and AI algorithm engineers can all operate sandboxes conveniently.

The v0.3.0 release answered one question: once a Sandbox is handed to an Agent, how do you upgrade it, resize it, and account for it. Its roadmap closed with two promises — network governance through TrafficPolicy / SecurityProfile, and a security identity integration that keeps platform credentials out of Agent code.

OpenKruise Agents v0.6.0 delivers both, and adds the operational machinery around them: 283 merged pull requests and 27 new contributors since v0.3.0. This post covers four capabilities in depth:

L4 and L7 network access control plus token management — destination reachability, per-request inspection, MCP tool-level ACLs, and credential injection from Kubernetes Secrets;

SandboxUpdateOps with CheckpointRestore and paused-instance upgrades — batch-upgrade a hibernated fleet without waking it by hand, and without losing the filesystem;

PoolAutoscaler — declarative warm pool capacity with watermarks and cron schedules;

Automatic pause and resume — probe-driven idle detection, scheduled wake, and wake-on-traffic, so idle Sandboxes stop consuming CPU.

Together these deliver three kinds of customer value: security hardening, operational capability, and cost optimization. The post is organized in three parts: the architecture changes that made them possible, a deep dive into the four capabilities, and the value summary and roadmap.

Part One: What Changed in the Architecture

If v0.3.0 was a workload layer, v0.6.0 is a workload layer with its own data plane. Traffic inspection, policy distribution, and identity all became first-class components rather than external add-ons. The overall architecture:

OpenKruise Agents v0.6.0 overall architecture

From four components to six. v0.6.0 adds traffic-control-plane and traffic-proxy:

Component

Role in v0.6.0

sandbox-manager

Stateless backend serving the E2B-compatible northbound API; owns sandbox lifecycle, quota, and route & traffic-token orchestration

sandbox-controller

Reconciles Sandbox, SandboxSet, SandboxClaim and warm pools; hosts admission webhooks; drives the automatic pause/resume controller

sandbox-gateway

Data plane built on Envoy Go filters. The ingress gateway proxies user traffic (host/path routing, access tokens, wake-on-traffic); the egress gateway enforces policy-driven L7 egress

traffic-control-plane (new)

istiod distributes xDS configuration to gateways and in-pod proxies; the Egress Policy Enforcer (EPE) applies L7 policy, MCP ACL, and token injection on egress traffic

agent-runtime

Sidecar providing E2B envd-compatible command, file, and port (49983) operations

traffic-proxy (new)

Sidecar acting as the in-pod L4 enforcement point for egress traffic policies

Two consequences are worth internalizing. First, egress is no longer the Pod’s own business — it is captured in-pod, routed through an egress gateway, and decided by a policy engine that sees the full request, not just the socket. Second, the ingress gateway can wake a sandbox, which is what makes wake-on-traffic possible without an external poller.

what makes wake-on-traffic possible without an external poller.

The CRD count doubled from 6 to 12, adding PoolAutoscaler, TrafficPolicy, GlobalTrafficPolicy, SecurityProfile, and GlobalSecurityProfile — a multi-tier control system spanning cluster, tenant, and sandbox scopes.

Part Two: The Four Core Capabilities

2.1 Network Access Control and Token Management

This is the largest change in the release. It answers three questions: what a Sandbox may access, who may access a Sandbox, and — the question most platforms get wrong — who actually holds the upstream API key.

The data plane uses a two-stage traffic takeover, node-side L4 plus central L7, balancing performance and function:

Sandbox application container -> traffic-proxy (in-pod L4 capture; Deny hits are blocked in place) -> egress gateway (central gateway chosen by egress routing) -> Envoy external processing (ext_proc) -> Egress Policy Enforcer decision (L7 / MCP ACL / credential injection) -> external service, or a blocked response

L4 filtering completes node-side with very low latency; L7 deep inspection is centralized in the gateway for unified operations, observability, and security audit. The split is deliberate: L4 denials are dropped before traffic reaches the gateway, and L7 can never override an L4 denial.

L4: is this destination reachable at all

TrafficPolicy selects caller Pods and holds an ordered egress rule list; peers support fqdn, cidr, service, and workload. Three semantics matter more than the syntax:

Once a policy selects a Pod, egress traffic matching no allow rule is denied — a policy is a whitelist once active;

Rules with a missing peer field are skipped by the compiler and do not form a default deny — a typo in to silently weakens the policy;

Always allow the cluster DNS Service, or every FQDN rule will look broken.

apiVersion: agents.kruise.io/v1alpha1 kind: TrafficPolicy metadata: name: sandbox-egress spec: selector: matchLabels: app: traffic-sample egress: rules: - action: allow to: - service: {name: kube-dns, namespace: kube-system} - action: allow to: - fqdn: api.example.com - action: reject ports: [{protocol: TCP, port: 443}] to: - cidr: 10.0.0.0/8

L7: what is this specific request allowed to do

SecurityProfile matches HTTP requests and applies actions. block and bypass are terminating; mcpToolPolicy, headerManipulation, tokenTransformation, and audit are not. The highest-value use for Agent workloads is SSRF protection — prompt-injected code almost always tries internal services and cloud metadata endpoints:

apiVersion: agents.kruise.io/v1alpha1 kind: GlobalSecurityProfile metadata: name: cluster-baseline spec: selector: matchLabels: {app: traffic-sample} priority: 50 # cluster baseline deliberately ordered before tenant profiles rules: - name: deny-metadata-endpoint match: - domains: - metadata.internal - "*.internal.company.com" actions: block: statusCode: 403 body: '{"error":"access to internal services is forbidden"}'

MCP tool-level ACL

When an Agent talks to an MCP Server, domain-level control is too coarse; the real risk is a specific tool. mcpToolPolicy governs only JSON-RPC tools/call and matches params.name exactly. Use defaultAction: deny for a whitelist.

Credential injection: your API key never enters the Sandbox

Credentials live in Kubernetes Secrets; the egress proxy resolves them per request and rewrites the outbound request as it leaves the cluster. Workloads and Agent code only ever see their own placeholder header.

apiVersion: agents.kruise.io/v1alpha1 kind: SecurityProfile metadata: name: example-api-credentials namespace: agent-demo spec: selector: matchLabels: {app: agent} rules: - name: inject-example-api-key match: - domains: [api.example.com] schemes: [https] paths: [{type: Prefix, value: /v1/}] actions: tokenTransformation: type: ApiKey failStrategy: Block # fail closed when the credential is missing credentialRef: secret: name: example-api-credentials apiKey: targetHeaders: names: [authorization] value: template: "Bearer {{ .Token }}"

Because resolution happens proxy-side, credentials never appear in Pod specs, environment variables, or Sandbox metadata, and rotating a Secret takes effect without restarting the Sandbox.

Core performance metrics

The traffic-control data plane is built around sandbox-gateway and has been validated at production scale. Reference gateway specifications (Alibaba Cloud ACS Agent Sandbox default deployment):

Metric

Value / specification

Per-replica instance size

4 vCPU / 8 GiB (default)

Default replicas

3

Per-replica ingress+egress bandwidth

1.5 Gbit/s

Theoretical throughput ceiling

1.5 Gbit/s / 2 (bidirectional) / 8 ≈ 93.75 MB/s

Recommended business traffic

≈ 90 MB/s per replica

L4 filtering location

Node-side traffic-proxy, completed in kernel space with very low latency

Metrics endpoint

Port 9902 (Prometheus format)

Tuning note

envoy.concurrency must be adjusted together with resources.cpu

Note: these are theoretical upper bounds, not committed service metrics. Observe request duration via the gateway log duration_ms field and watch for 502s or packet loss in production.

In production at leading customers, this sandbox and traffic-control stack supports:

MiniMax (MaxClaw): custom template pre-warming plus MicroVM for 20–40 ms instance provisioning, up to 15,000 sandboxes/minute of elastic supply.

Kimi (Moonshot AI): tens of thousands of concurrent requests at peak with a hundred-thousand-Pod backend; minute-level thousand-node scale-out.

Li Auto: MicroVM hardware isolation with fine-grained east-west and north-south traffic control; about 800 employees in daily use, planned to grow to tens or hundreds of thousands of Agents.

Guardrails you must not skip. Controls cover only gateway-routed traffic; containers running as root or with NET_ADMIN / SYS_ADMIN can bypass in-pod ACLs. Combine these policies with Kubernetes NetworkPolicy, host firewalls, or cloud security groups — this is defense in depth, not a wall.

2.2 SandboxUpdateOps: CheckpointRestore and Paused-Instance Upgrades

v0.3.0 offered a single mode, Recreate: destroy and rebuild the Pod, losing memory, IP, and the writable rootfs layer. v0.6.0 adds the two most-requested capabilities — a mode that preserves rootfs, and the ability to include paused Sandboxes in batch upgrades.

Mode

Rootfs

Memory / IP

Change business image

Recreate (default)

Lost

Lost / changes

Supported

CheckpointRestore

Preserved (snapshot before, restore after)

Lost / changes

Not supported

CheckpointRestore trades image flexibility for data continuity and is the standard way to roll sidecars such as agent-runtime or traffic-proxy: user data untouched, no backup scripts. The new stateFilter lets the controller wake each paused Sandbox, upgrade it, and re-pause it while `spec.paused` remains true — the fleet ends in the same shape it started, so one upgrade never quietly turns a hibernating pool into a bill.

apiVersion: agents.kruise.io/v1alpha1 kind: SandboxUpdateOps metadata: name: upgrade-with-paused namespace: default spec: stateFilter: states: [Running, Paused] # include paused instances in the upgrade selector: matchLabels: agents.kruise.io/sandbox-template: my-sandbox-pool updateStrategy: maxUnavailable: 10% patch: spec: containers: - name: sandbox image: my-registry/sandbox-image:v2

2.3 Warm Pool Autoscaling

Warm pools are why OpenKruise Agents can claim a Sandbox in sub-second time — and why the platform bill keeps rising at night when nobody is using it. PoolAutoscaler turns pool capacity from “a cron job running kubectl scale” into a declarative policy: it rewrites SandboxSet.spec.replicas, providing watermark feedback based on available capacity plus time-driven cron overrides.

apiVersion: agents.kruise.io/v1alpha1 kind: PoolAutoscaler metadata: name: sandbox-pool-scaler namespace: default spec: scaleTargetRef: {kind: SandboxSet, name: sandbox-pool} minReplicas: 2 maxReplicas: 50 capacityPolicy: targetAvailable: "50%" # keep half the capacity available to absorb claim bursts tolerance: "10%" scaleUp: {stabilizationWindowSeconds: 60} # scale up fast scaleDown: {stabilizationWindowSeconds: 300} # scale down slow cronPolicies: - name: weekday-peak timezone: Asia/Shanghai schedule: "30 8 * * 1-5" # scale up before the morning peak targetReplicas: 30 - name: weekday-offpeak timezone: Asia/Shanghai schedule: "0 20 * * 1-5" # scale down at night to save cost targetReplicas: 5

Asymmetric stabilization windows (fast scale-up, slow scale-down) are the correct default for warm pools: users feel shortages immediately and never notice slow scale-down. Cron triggers take priority and fire before the burst arrives.

A complementary cost capability is in-place memory resize at claim time. A warm pool does not need to hold instances at peak business memory: the pool can warm at a small uniform size, and when a SandboxClaim claims an instance, an in-place resize adjusts the main container’s CPU/memory to what that task actually needs (in-place resize of a running Pod requires Kubernetes 1.33+; the SandboxInPlaceResourceResize feature gate is enabled by default). This sacrifices some delivery speed — an in-place resize is slightly slower than claiming an instance that already fits — but remains far faster than cold-starting a new Sandbox, and it sharply cuts warm pool holding cost: the pool holds one fleet of small instances instead of a large fleet per business size.

2.4 Automatic Pause and Resume

To Kubernetes, a claimed Sandbox with an active session and one nobody has touched for four hours look identical. Automatic pause/resume makes that difference observable and acts on it. It is configured through two top-level SandboxSet fields, probes and autoPausePolicy, with three capabilities:

Idle-driven pause: a probe (exec only) reports activity periodically; sustained idle beyond thresholdDuration pauses the Sandbox;

Scheduled wake: for workloads with a timetable (e.g. daily knowledge organization at midnight, a weekly report on Friday afternoon), a probe prints the next activation time and the controller wakes the Sandbox ahead of it;

Wake-on-traffic: the first request after pause wakes the Sandbox instead of failing — the key to making pause safe for user-facing scenarios.

The following SandboxSet enables both idle pause and scheduled wake:

apiVersion: agents.kruise.io/v1alpha1 kind: SandboxSet metadata: name: openclaw-sbs namespace: default spec: replicas: 4 probes: - name: Active containerName: sandbox periodSeconds: 30 timeoutSeconds: 5 exec: command: - sh - -c - | # Example only; define the idle state per your business. if openclaw sessions list --active 900 2>/dev/null | grep -q .; then echo "active" else echo "inactive" fi - name: Cron containerName: sandbox periodSeconds: 30 timeoutSeconds: 5 exec: command: - sh - -c - | # Example only: next run time of the earliest enabled task # (daily 00:00 knowledge organization, Friday 15:00 weekly report) NEXT_MS=$(openclaw cron list --json 2>/dev/null \ | jq -r '[.[] | select(.enabled != false) | .nextRunAtMs] | map(select(. != null)) | sort | .[0]') if [ -n "$NEXT_MS" ] && [ "$NEXT_MS" != "null" ]; then echo $((NEXT_MS / 1000)) # nextRunAtMs is milliseconds; divide by 1000 else echo "none" # no pending task: no wake schedule fi autoPausePolicy: pause: whenProbedIdleState: # idle-driven pause probe: Active messageRegex: "^inactive$" thresholdDuration: 10m # pause after 10 consecutive idle minutes resume: whenProbedScheduleTime: # scheduled wake probe: Cron timeFormat: unix leadTime: 5m # wake 5 minutes before the task template: spec: containers: - name: sandbox image: <YOUR_IMAGE>

The Cron probe fetches the next scheduled task time every 30 seconds: the example filters enabled tasks from openclaw cron list, sorts by nextRunAtMs, takes the earliest (the nearer of the daily midnight knowledge organization and the Friday afternoon weekly report), and prints it as Unix seconds after dividing milliseconds by 1000; with no pending task it prints none and the controller creates no wake schedule. leadTime: 5m wakes the Sandbox 5 minutes before the task.

Three behaviors are universal: requests must pass authentication (unauthorized requests return 401 and the Sandbox stays paused — wake-on-traffic is not an unauthenticated doorbell); a successful wake can accompany a failed request (the waking call may return 503; check Sandbox.status.phase and retry); a failed or timed-out probe never triggers pause or wake — the safe side.

from e2b_code_interpreter import Sandbox sandbox = Sandbox.create( template="code-interpreter", timeout=600, lifecycle={"on_timeout": "pause", "auto_resume": True}, ) host = sandbox.get_host(3000) # first request triggers wake-on-traffic

Part Three: Customer Value and Roadmap

Value dimension

What v0.6.0 changes

Security hardening

L4 whitelist + L7 request inspection + MCP tool whitelist + cluster-level baselines; credentials resolved proxy-side from Secrets and rotatable without restart; short-lived rotatable JWT traffic access tokens

Operational capability

stateFilter covers Running and Paused; CheckpointRestore preserves rootfs; PoolAutoscaler makes capacity declarative; automatic pause turns idle detection into a probe

Cost optimization

Idle Sandboxes pause on probe evidence and wake on schedule or first request; pools scale down at night and up before peaks; paused fleets still receive upgrades, avoiding data loss and needless wake cost

Roadmap: community capability building. The next phase emphasizes co-building with other open-source communities rather than a closed in-house stack:

Open-source checkpoint with the Kata community — based on the PodCheckpoint API introduced in Kubernetes 1.37, removing dependence on platform-specific snapshot backends;

On-demand mounting for open-source storage — MinIO and similar storage mounted on demand through the existing on-demand mounting framework;

Integration with open-source identity systems — for example Keycloak, so platform authentication reuses your existing identity provider.

Getting Involved

OpenKruise Agents is built entirely in the open. If you run Agent Sandboxes on Kubernetes, the fastest way to help shape the roadmap is to file an issue or join the community meeting. v0.6.0 is available on GitHub Releases and major image registries.

Project GitHub: https://github.com/openkruise/agents

Documentation: https://openkruise.io/kruiseagents/

Slack: Kubernetes Slack #openkruise channel

Biweekly community meeting (Chinese): Thursdays 19:30 GMT+8