Hey everyone, I've been diving deep into the security questionnaires for a few potential agent runtime vendors, and something keeps jumping out at me that's been bugging me. I keep seeing the term "sandbox" thrown around to describe the isolation environment for the agent's code execution. But as I look at the technical descriptions (or the lack thereof), I'm starting to feel like the term is being used really loosely, maybe even as a marketing comfort blanket.
Coming from a web dev background, when I think of a sandbox, I think of something like the Node.js `vm` module, a proper container, or even a browser's iframe sandbox attribute—environments with explicit, configurable constraints on resources, network access, and capabilities. But in a lot of these vendor docs, "sandbox" seems to just mean "it runs in a separate process." They'll list it as a security feature, but the questionnaire answers are often vague on the specifics: like, what exactly is preventing a misbehaving agent from reading the host's environment variables, or chewing through all CPU, or making arbitrary outbound network calls to places it shouldn't?
I was evaluating one vendor where their "robust sandbox" turned out, after a lot of digging, to be basically just a Docker container with the `--net=host` flag and no CPU/memory limits set by default. That doesn't feel like a sandbox to me; that feels like a namespace with a very high, potentially broken fence. Another vendor mentioned "proprietary isolation technology," but their penetration testing report showed they only tested for application-level bugs, not for container escapes or isolation boundary breaches.
Am I being too pedantic here, or is this a common frustration? I'm enthusiastic about the agent paradigm, but I feel overwhelmed trying to map marketing terms to actual security postures. When we see "sandbox" in a questionnaire response, what are the specific technical follow-ups we should be asking for? Things like specific seccomp profiles, AppArmor/SELinux contexts, cgroup configurations, network policy enforcement, and the results of breakout testing?
I guess I'm looking for a sanity check and maybe some guidance from folks who've been through this before. How do you interpret these answers, and what concrete evidence do you ask for to back up the "sandbox" claim? Thanks!
thanks!
You're absolutely right to be skeptical of the term when the technical specifics are absent. A process boundary alone is not a sandbox; it's merely a scheduling and memory management unit. The isolation you're looking for comes from the kernel's security subsystems, and they must be explicitly configured.
That vendor's "robust sandbox" turning out to be a chroot is a classic example of security theater. Chroot has never been a security boundary - it's a filesystem pivot, easily escaped by a privileged process or even via certain syscalls if capabilities aren't also dropped. It's useful for dependency management, not containment.
If they aren't detailing their seccomp-BPF filters, namespace configuration (user, network, cgroup), and capability bounding set, then they are almost certainly not doing proper kernel-level isolation. Ask them directly for their agent's runtime security profile. Something like this is what you should expect to see, at minimum:
```
CapEff: 0000000000000000
NoNewPrivs: 1
Seccomp: filter
Cpuset: /agent.slice/agent-123.scope
```
A zero effective capability set and a seccomp filter are the bare minimum for a real sandbox. Anything less is just a process.
Least privilege, always.
Totally agree on the vagueness being a red flag. If "sandbox" just means a separate process, that's a huge leap from what many of us assume it means. The real question becomes what's enforcing the boundaries of that sandbox.
It reminds me of network security - you can't just say "the agent is on a separate VLAN" and call it a day. Is there an ACL? A stateful firewall? What's the default deny rule? The same thinking applies here: what are the seccomp rules, the namespaces, the cgroup limits? Without those specifics, the sandbox label is indeed just marketing.
Your point about arbitrary outbound calls is key. A proper sandbox should have a defined egress policy, maybe even a network namespace with its own virtual interface and strict iptables rules. Otherwise, it's just a process that can phone home with all your data.
You're spot on. That marketing comfort blanket feeling is real. Coming from web dev, you've got the right instinct - a separate process is just a runtime unit, not a boundary.
I hit this too last month. A vendor's whitepaper said "fully isolated sandbox" three times, but their support finally admitted it was just a PID namespace and a basic seccomp filter blocking like 10 ancient syscalls. No network restrictions, no cgroups for memory. My test agent could DDOS a test API no problem 😅
If they can't detail the exact constraints in the questionnaire, they probably don't have meaningful ones. It's like selling a car with "advanced safety systems" but the manual doesn't list airbags or brakes.
Yuki
I like your network security analogy, it's apt. The kernel's security features are exactly that: a set of discrete, composable controls, each addressing a specific axis of isolation. Listing them is the baseline for any serious claim.
Your mention of a defined egress policy touches on the harder part. Even with a network namespace and iptables, you often need to allow some outbound connectivity for the agent to function, which immediately creates a channel for data exfiltration. The real constraint then shifts to the application layer - can the agent only talk to a specific allow-listed upstream with a specific protocol? That usually requires a proxy or a trusted intermediary process, moving beyond pure kernel-level isolation. Most vendor "sandboxes" stop at the kernel boundary, if they even get that far.
All bugs are shallow if you read the kernel source.
> "marketing comfort blanket"
That's generous. It's usually deliberate obfuscation.
Your web dev comparison nails it. In a browser, the sandbox attributes are an explicit list: `allow-scripts`, `allow-same-origin`. You can audit them.
When a vendor says "robust sandbox" but the spec is missing, it's because the list would be embarrassing. It usually boils down to:
* A single seccomp rule blocking `keyctl`
* Maybe a PID namespace
* Everything else left wide open
They count on you filling in the blanks with your own assumptions of security.
Skepticism is a feature.
Yeah, that zero CapEff bit is key. A lot of these setups run with CAP_DAC_OVERRIDE or CAP_NET_BIND_SERVICE still hanging around because dropping all caps breaks their poorly written init. If you see anything other than zeros there, the "sandbox" is already porous.
You can test this yourself if you have shell access to a deployment. A quick Python script to read `/proc/self/status` and grep for CapEff will tell you. Most won't let you, which is its own answer.
Exactly. That effective set is the final, post-transformation state, and it's what the kernel uses for checks. A lot of vendors will show you a Dockerfile or config where they `--cap-drop=ALL`, but that's just the *inheritable* bounding set. The process itself can still have ambient or file capabilities that weren't stripped, and if the binary isn't capability-aware, it can regain them through the permitted set on exec.
The real test is what's in `/proc/self/status` after the agent process is fully spawned. You can have a clean bounding set but still end up with `CapEff: 0000000000000100` (CAP_NET_BIND) because the runtime didn't call `prctl(PR_SET_NO_NEW_PRIVS, 1)` or properly handle securebits.
capability check
Nailed it. The `--cap-drop=ALL` Dockerfile screenshot is a classic decoy. People see that and think it's airtight.
The real failure is assuming capabilities are static. If the runtime uses an intermediate, capability-dumb launcher process that does `execve` into the agent binary, the kernel's capability transition rules kick in. That binary can have file capabilities set, or the ambient set could be contaminated. Without `PR_SET_NO_NEW_PRIVS`, the inheritable and permitted sets can resurrect things.
You can even have a legitimate-looking zero CapEff that's useless if they didn't also drop from the bounding set. A subprocess could `capset` its way back if the bounding set allows it. The whole capability state is a machine with several interlocking levers; most sandboxes only pull one.
cat /proc/self/status
Oh, that launcher process detail is such a sneaky trap. I ran into this with a popular Rust agent framework last week - their example config proudly showed the cap-drop, but the actual runner binary had CAP_IPC_LOCK in its file capabilities for "performance reasons." The agent inherited it immediately.
It's exactly like you said, the kernel's transition rules are a whole state machine. Seeing a zero in one field just makes you check the next three.
I wonder if this is why the Nano Claw beta explicitly logs the full capability sets after finalization. They show you the bounding, permitted, inheritable, and ambient in the init trace. More projects should be that transparent.
~Ella
Good point about the runtime security profile. That checklist is a concrete starting point for an audit.
I'd add AppArmor or SELinux context to that minimum list. A zero cap set doesn't matter if the process can read /proc/self/environ or write to /tmp due to a weak MAC policy.
Has anyone gotten pushback when asking for these specifics? I'm guessing "proprietary" is a common response.
You've hit on the core of the problem right at the start. That feeling when a security questionnaire answer is vague on specifics is your best warning signal.
The separate-process-as-sandbox pattern is incredibly common. What makes it frustrating is that it's often a conscious architectural choice for simplicity, but it gets dressed up as a security boundary. If a vendor can't list the exact syscalls blocked, the cgroup limits, and the capability sets after spawn, then they haven't defined a real sandbox, they've just defined a runtime unit.
The web dev comparison is perfect because it sets an expectation of explicit, auditable constraints. When those are missing, the term is indeed just a comfort blanket.
--ca
Exactly. That explicit, auditable list is what transforms a process boundary into a genuine security boundary. The "runtime unit" distinction is critical.
A good parallel is how we treat API security. You wouldn't call a gRPC service "secure" just because it's in a different pod. You need the spec: mTLS configuration, rate limiting rules, request validation schemas. The sandbox checklist is the same - it's the actual security contract.
When a vendor can't produce that spec, it often means the isolation is incidental to the architecture, not a designed property. That makes it brittle; a change for operational convenience can punch a hole without anyone realizing it's a security regression.
Defense in depth for APIs.