Running AutoGen agents in production means dealing with multiple API keys—OpenAI, Anthropic, maybe Azure, plus any tools with their own credentials. Hardcoding these in your `config_list` is a fast track to a breach. Auditors will fail you on sight.

The core problem is separating secret storage from the agent runtime code. You need a method that works both locally for development and in a deployed environment, without changing your agent code.

Here's my practical approach:

1. **Environment Variables for Core LLM Keys:** Never store API keys in your code. Use a `.env` file locally (added to `.gitignore`) and your platform's secret manager (e.g., AWS Secrets Manager, HashiCorp Vault) in production. Your `config_list` generation should read from these.
2. **A Centralized `ConfigBuilder`:** Create a single function that builds your `config_list` dynamically. It pulls base URLs and API keys from the environment, and you can programmatically switch between providers.

Example structure:

```python
# config_builder.py
import os

def get_config_list(model_type="gpt-4", api_keys=None):
"""
model_type: hint for which provider/key to use
api_keys: dict, defaults to reading from os.environ
"""
if api_keys is None:
api_keys = {}

openai_key = api_keys.get("OPENAI_API_KEY", os.getenv("OPENAI_API_KEY"))
anthropic_key = api_keys.get("ANTHROPIC_API_KEY", os.getenv("ANTHROPIC_API_KEY"))

config_map = {
"gpt-4": [
{
'model': 'gpt-4',
'api_key': openai_key,
'base_url': "https://api.openai.com/v1" # Optional: for Azure or other endpoints
}
],
"claude-3": [
{
'model': 'claude-3-opus-20240229',
'api_key': anthropic_key,
'base_url': "https://api.anthropic.com/v1"
}
]
}
return config_map.get(model_type, [])
```

Then in your agent code:
```python
from config_builder import get_config_list

agent_config = get_config_list(model_type="gpt-4")
assistant = autogen.AssistantAgent("assistant", llm_config={"config_list": agent_config})
```

For tool credentials (e.g., database passwords), use the same pattern but inject them into the agent's `llm_config` under `extra_args` or manage them via a separate, secured tooling layer.

Common gaps auditors flag:
* Keys logged in application logs (mask them).
* Lack of key rotation procedures.
* Over-permissioned service accounts for tools the agents call.
* No audit trail for which agent used which key.

What's your deployment stack? The specific secret manager you use changes the implementation, but the pattern stays the same.

-- mike