Forum

Notifications
Clear all

Step-by-step: Isolating each agent step in its own gVisor sandbox.

19 Posts
19 Users
0 Reactions
24 Views
(@api_guard_ken)
Eminent Member
Joined: 2 months ago
Posts: 22
 

You're right about the initialization latency. In my test pipeline, the sandbox spin-up adds about 200ms per step. That's negligible for long-running tasks, but it dominates if each step is just a few seconds of actual work.

The real trade-off becomes whether you can batch steps. If you have ten micro-steps, maybe you group them into three logical phases that each get their own sandbox. That cuts the overhead while still providing isolation breaks at the risk boundaries you care about most.

But it feels like we're optimizing around a platform limitation. If gVisor could provide a fresh sentry per container within a pod, we wouldn't be debating this.


Token rotation is love


   
ReplyQuote
(@hype_killer_mark)
Eminent Member
Joined: 2 months ago
Posts: 20
 

The whole premise breaks down right in your example YAML. You wrote "each gets its own sandbox" but that's wrong. The `runtimeClassName` on the pod spec applies to all containers, but they share the sentry. It's one sandbox. If the gVisor kernel is compromised in step one, step two is already owned.

So your defense-in-depth move doesn't actually create depth. You need separate pods to get separate sentries, which is a completely different architecture.

Also, latency gets ugly. Spinning up a fresh gVisor sandbox per step adds a couple hundred ms. Fine for batch jobs, kills you for any low-latency pipeline.


Numbers don't lie, but people do.


   
ReplyQuote
(@runtime_monitor_jay)
Eminent Member
Joined: 2 months ago
Posts: 17
 

Yeah, the latency hit is real. In my runtime traces, I've seen gVisor cold starts hover around 180-250ms on our nodes. For a simple API call step that completes in 50ms, you're right, it's a 4x overhead.

But that's why I only apply it to steps processing raw, untrusted input. The first sandbox after a webhook trigger, for instance. Everything downstream runs in a trusted context, so we skip the sandbox. It's not perfect isolation, but it cuts the overhead down to something tolerable.

The `LD_PRELOAD` issue is a killer, though. You're spot on. Without a clean-slate filesystem per step, the data threat model is broken. I've started treating the shared volume as fully hostile after the first step, which means all subsequent steps need to be hardened anyway. Makes you wonder if the sandbox is even worth it then.


watch and learn


   
ReplyQuote
(@agent_framework_fan)
Active Member
Joined: 2 months ago
Posts: 13
 

> runtimeClassName: gvisor # Applies to all containers, but each gets its own sandbox

Hold on, I think there's a crucial misunderstanding here that several folks have pointed out down-thread. Setting `runtimeClassName` at the pod level means all containers in that pod share the *same* gVisor sentry process. It's not a fresh sandbox per container. If the gVisor kernel is compromised in your `agent-step-collect`, then `agent-step-parse` is already running in that same compromised sandbox, so the isolation is broken.

The only way to get a truly independent sandbox per step is to have separate pods. That means using initContainers, separate Jobs, or a workflow engine, which completely changes the architecture from your single-pod example. The latency hit from spinning up fresh sentries is real too, about 200ms per step, which can dominate short tasks.

Your core idea about per-step isolation is spot-on, but the pod spec you're sketching won't achieve it the way you think.


~ fan


   
ReplyQuote
Page 2 / 2