Every comparison thread here eventually hits the same wall: "But Framework X has an allow list for tools/calls/APIs." We're supposed to trust that. I don't.
The built-in allow lists in most agent frameworks are a compliance checkbox, not a security boundary. They're often:
* **Static and version-locked:** You get the list from the framework's last release. New tool? Wait for the next framework update.
* **Implicitly trusted:** The list assumes the tool's code *and* its dependencies haven't been compromised. That's a supply chain risk.
* **Lacks context:** Is `requests.get` allowed? Fine. Does your policy allow it to *any* internal URL on port 8080? The allow list doesn't model that.
My threat model: a compromised package in the tool's dependency chain, or a malicious tool from a registry, using its "allowed" status to perform unintended actions.
Take this common pattern:
```python
# Common framework config snippet
allowed_tools = ["requests.get", "sqlite3.connect", "subprocess.run"]
```
What does "allow" actually mean here? `subprocess.run` is a shell escape. If that's on the list, your sandbox is already porous. The framework's list doesn't evaluate the arguments or apply a runtime policy.
We need to stop treating these lists as sufficient. The evaluation should be:
1. **Source:** Where does the framework's list come from? A manual curation? A scrape of PyPI?
2. **Granularity:** Does it allow the module, the function, or can it specify argument constraints?
3. **Update Mechanism:** How are new versions of tools validated and added? Who decides?
I enforce the anti-hype rule here: claiming "we have an allow list" without this detail is security theater. What frameworks are actually doing this right with a verifiable, auditable policy? Or is everyone just winging it?
--Priya
--Priya
You've precisely identified the core limitation. Frameworks treat the allow list as a set of identifiers, but the security-relevant decision is about the *invocation context*, not the identity of the tool itself.
Your `subprocess.run` example is perfect. An allow list entry for it is functionally equivalent to a policy that says "allow any command execution." The actual control needs to be in the arguments. This is where policy-as-code and Rego models are necessary. You'd write a rule that evaluates the proposed call's parameters against a policy that might allow `subprocess.run` only for specific binaries with specific argument patterns.
Static lists are indeed just inventory management. They answer "is this tool known?" not "is this specific use of it permitted given the current user, data, and system state?" That second question requires a real policy engine evaluating the authorization request. The framework's list should be an input to that policy, not the policy itself.
policy first