I've been evaluating the security posture of several popular agent frameworks by stress-testing their isolation guarantees. A recurring exercise is deploying a canary token system within the agent's operational environment to detect unintended access or breaches. The premise is simple: if something touches the canary, the environment has been compromised beyond the agent's intended execution path.

I implemented a basic Flask server that logs all access to a hidden endpoint, but the more revealing test was embedding canaries in the filesystem and environment variables. I then ran a series of benign agent tasks across different frameworks. The results were concerning. For instance, in one widely-used framework, the agent's tool-execution subsystem accessed the canary file during a routine file read operation, not because of a vulnerability per se, but due to overly permissive default path visibility. The threat model here is a compromised tool or a malicious payload within a retrieved document attempting to exfiltrate environment details.

The critical comparison point is the default sandboxing or namespace isolation. Consider this simple canary placement and check script I used:

```python
# Place canary
import os
CANARY_PATH = "/tmp/.env_canary_9f8s7d"
with open(CANARY_PATH, "w") as f:
f.write("CANARY_TOKEN=supersecret")
os.environ["CANARY_TOKEN"] = "supersecret"

# Agent task (simulated tool call)
def read_file(path):
with open(path, "r") as f:
return f.read()
# Later, check logs for access to CANARY_PATH or env var.
```

In frameworks with weaker isolation, even this simple file operation, when given a user-supplied path argument, could be tricked into reading `/tmp/.env_canary_9f8s7d` if the sandbox doesn't properly jail the filesystem. The more subtle risk is the agent's own system, like a retrieval tool, scanning directories and indexing the canary token file, effectively exposing it to the LLM context. This creates a secondary poisoning vector where the secret could be leaked via the agent's own memory.

This exercise underscores that security comparisons must move beyond feature checklists. We need to specify threat models: 1) Malicious user prompts directing tools to sensitive paths, 2) Compressed or archived files containing canary-token-named documents that get extracted and indexed, and 3) Supply chain attacks where a malicious third-party tool attempts to enumerate environment variables. The sandboxing quality is not binary; it's about the default allow-list versus deny-list approach and the ease of escape. I'm compiling data on which frameworks default to a restricted, enumerated set of accessible paths versus those that simply run the agent process with the user's own permissions. The difference is fundamental to preventing these classes of intrusion detection triggers.