Every comparison thread here eventually hits the same wall: "But Framework X has an allow list for tools/calls/APIs." We're supposed to trust that. I don't.
The built-in allow lists in most agent frameworks are a compliance checkbox, not a security boundary. They're often:
* **Static and version-locked:** You get the list from the framework's last release. New tool? Wait for the next framework update.
* **Implicitly trusted:** The list assumes the tool's code *and* its dependencies haven't been compromised. That's a supply chain risk.
* **Lacks context:** Is `requests.get` allowed? Fine. Does your policy allow it to *any* internal URL on port 8080? The allow list doesn't model that.
My threat model: a compromised package in the tool's dependency chain, or a malicious tool from a registry, using its "allowed" status to perform unintended actions.
Take this common pattern:
```python
# Common framework config snippet
allowed_tools = ["requests.get", "sqlite3.connect", "subprocess.run"]
```
What does "allow" actually mean here? `subprocess.run` is a shell escape. If that's on the list, your sandbox is already porous. The framework's list doesn't evaluate the arguments or apply a runtime policy.
We need to stop treating these lists as sufficient. The evaluation should be:
1. **Source:** Where does the framework's list come from? A manual curation? A scrape of PyPI?
2. **Granularity:** Does it allow the module, the function, or can it specify argument constraints?
3. **Update Mechanism:** How are new versions of tools validated and added? Who decides?
I enforce the anti-hype rule here: claiming "we have an allow list" without this detail is security theater. What frameworks are actually doing this right with a verifiable, auditable policy? Or is everyone just winging it?
--Priya
--Priya