I'm using AutoGen's `UserProxyAgent` with Azure OpenAI. I've set up separate `llm_config` objects for different tasks, each with its own Azure endpoint and API key defined in the `config_list`. The problem is, agents seem to be using credentials from one config for tasks that should be using a different, more restricted set.
My understanding was that the `llm_config` attached to an agent defines its scope. But in practice, when an agent starts a nested conversation or delegates, the credentials appear to leak across the agent boundary. This defeats the entire point of scoping credentials to the minimal necessary permissions for a task.
What I expected:
* Agent A (Financial Analyst) uses `llm_config_A` with key for finance endpoint.
* Agent B (Code Auditor) uses `llm_config_B` with key for code analysis endpoint.
* If Agent A needs help from B, it delegates. Agent B uses **its own** credentials (`llm_config_B`).
What I'm observing:
* The delegation seems to carry forward the original caller's credentials or some merged set.
This is a critical failure in isolation. If I'm modeling this correctly, the threat model breaks down completely:
* A compromised or malicious sub-agent could exfiltrate credentials intended for a different agent's scope.
* Tasks that should be limited to a low-privilege endpoint can potentially access high-privilege ones.
Has anyone dug into the actual credential flow in these delegation chains? Is there a central credential cache I'm missing? My current workaround is to run separate processes, which is heavy.
Key questions:
* Is this a bug in my configuration, or a fundamental flaw in how the framework handles credential isolation?
* What's the actual attack surface here? Credential exposure across trust boundaries seems likely.
* Are there any established patterns for true credential scoping in AutoGen, or do I need to fork and implement agent-specific credential injection at the call site?
If it's not in the threat model, it's not secure.