I've been prototyping a secure inference pipeline where the model itself needs a specific API key to call an external enrichment service. Hardcoding was out of the question, and environment variables felt too broad for the multi-tenant setup. HashiCorp Vault's precise leasing and revocation seemed like the right fit. Here's the pattern I landed on after a few iterations.

The core idea is to use Vault's Kubernetes authentication method if your agents are running in pods, or the AppRole method for other orchestration environments. The agent fetches a short-lived secret at runtime, caches it in memory only, and renews the lease as needed. This avoids secret sprawl.

Here's a minimal Python example using the `hvac` client for an agent that needs a `WEATHER_API_KEY`. We use AppRole here for clarity.

```python
import hvac
import os

def get_secret_from_vault():
# Authenticate to Vault
client = hvac.Client(url=os.environ['VAULT_ADDR'])

# Using AppRole. In K8s, you'd use kubernetes auth instead.
role_id = os.environ['VAULT_ROLE_ID']
secret_id = os.environ['VAULT_SECRET_ID']
client.auth.approle.login(role_id=role_id, secret_id=secret_id)

# Read the secret. Vault path is configurable.
secret_response = client.secrets.kv.v2.read_secret_version(
path='agent-secrets/weather-service'
)
api_key = secret_response['data']['data']['api_key']

# Use the key for your model's external call.
return api_key
```

Critical safety points:
* The `VAULT_ROLE_ID` and `VAULT_SECRET_ID` are injected via the pod spec or runtime environment. They are themselves short-lived.
* The Vault policy for this role must be tightly scoped—only read access to the specific path needed.
* Never log the secret or the response. Implement a local memory cache with lease renewal logic to avoid hitting Vault on every inference request.
* Consider wrapping the secret fetch in a circuit breaker. If Vault is unreachable, your agent should fail securely, not default to a fallback key.

An unsafe pattern I've seen: pulling the secret at agent startup and storing it in a global variable that gets serialized if the model checkpoint is saved. This could lead to accidental secret inclusion in model artifacts. Always fetch the secret in the runtime context where it's used, and ensure your serialization methods (like `torch.save` or `pickle`) never include that context.