Oleksandr Kuzmenko, EditorPublished 9 min readAI-assisted · editor-reviewed
TL;DR
- Agent Substrate decouples session state from running pods, allowing Kubernetes to freeze idle agents without losing context.
- Resume latencies clock under 400ms, avoiding 20-second serverless cold starts while cutting compute idle bills by up to 95%.
- Agent code executes strictly inside gVisor or Cloud Hypervisor microVM sandboxes with default-deny networking.
Key facts
- Typical Agent Idle Time
- 90% - 95% (Google self-reported)
- Session Resume Latency
- < 400 ms (project self-reported)
- Activation Throughput
- > 1,500 activations/sec (project self-reported)
- Licensing
- Apache-2.0
- Isolation Engines
- gVisor user-space kernel and Cloud Hypervisor microVMs
Solving the 95% Idle Time Tax in Kubernetes
AI agents spend the vast majority of their operational lifespan idle. According to data published by Google Cloud, interactive agent sessions wait between 90% and 95% of the time for model completion tokens, external API round-trips, or human input. Running persistent Kubernetes Pods for every active user leads to prohibitive infrastructure bills once deployments scale past 10,000 active users. Conversely, relying on standard serverless scale-to-zero architectures introduces cold starts upwards of 20 seconds.
To address this infrastructure gap, Google has donated Agent Substrate, gVisor, and OpenRL to the Cloud Native Computing Foundation (CNCF) under Sandbox status. The stack establishes a vendor-neutral standard for orchestrating untrusted agent code on any conformant Kubernetes cluster.
Decoupling State via Actors and Pre-Warmed Pools
Agent Substrate decouples execution state from the underlying virtual machines using an Actor-Worker pattern:
- Actors: Represent individual session states, encapsulating filesystem diffs, environment variables, in-memory allocations, and egress networking policies.
- Workers: Pre-warmed Kubernetes Pods residing in a shared
WorkerPool. A single worker can sequentially host multiple isolated actor sessions. - atenet-router: A Layer-7 ingress proxy that intercepts inbound requests. If the targeted actor is suspended, the router buffers the payload in memory, signals the Substrate controller, and attaches the actor to an available worker.
Project benchmarks report session restore latencies below 400 milliseconds, support for over 1,500 activations per second per cluster, and roughly a 10x improvement in container density over traditional Pod allocations.
Hardened Sandboxing and Production Readiness
Untrusted code generated by coding agents executes inside gVisor application kernels or Cloud Hypervisor microVMs rather than unshielded root containers. System calls are trapped in user space, network traffic is default-deny, and credentials are dynamically injected at an egress gateway accompanied by short-lived SPIFFE X.509 certificates.
On Google Kubernetes Engine (GKE), version 1.37 serves the necessary APIs by default, while version 1.36 requires beta API flags. Note that while Google's blog describes Substrate as a 1.0 milestone, the repository README continues to flag the architecture as early-stage without strict backward compatibility guarantees.
When to use
- Deploy when running interactive, chat-based, or multi-step coding agents that spend significant time awaiting human confirmation.
- Use for high-frequency RL evaluation loops where spawning thousands of ephemeral Pods threatens etcd stability.
When NOT to use
- Do not use for strictly batch, non-interactive agent jobs where pods run continuous workloads to completion without external waiting.
- Avoid using in production clusters where you cannot tolerate pre-1.0 architectural changes or lack gVisor kernel compatibility.
What to do today
- Review your agent cluster metrics to calculate the ratio of idle API waiting time versus active compute execution.
- Explore the Agent Substrate architecture at ate.dev and test sandboxed actor lifecycles on a staging Kubernetes 1.37 cluster.
- Evaluate gVisor runtime classes for existing coding agents to isolate untrusted shell and filesystem executions.
Step names are taken from What to do today.
Step names are taken from What to do today. Review your agent cluster metrics to calculate the ratio of idle API waiting time versus active compute execution.. Explore the Agent Substrate architecture at ate.dev and test sandboxed actor lifecycles on a staging Kubernetes 1.37 cluster.. Evaluate gVisor runtime classes for existing coding agents to isolate untrusted shell and filesystem executions.
- Review your agent cluster metrics to calculate the ratio of idle API waiting time versus active compute execution.
- Explore the Agent Substrate architecture at ate.dev and test sandboxed actor lifecycles on a staging Kubernetes 1.37 cluster.
- Evaluate gVisor runtime classes for existing coding agents to isolate untrusted shell and filesystem executions.