We spend countless cycles here debating prompt injection, training data poisoning, and model alignment—worthy topics, to be sure. But I see a far more immediate and pedestrian threat vector being waved through the gates with minimal scrutiny: the plugin architecture everyone is rushing to implement.
The prevailing assumption seems to be that if the core LLM runtime is FedRAMP-authorized or sits within an IL5 boundary, then the plugins it calls are merely "tools," and their risks can be managed with simple allow-lists. This is a catastrophic failure of threat modeling. The agent *is* the sum of its capabilities. Granting an LLM the ability to call a plugin is granting that plugin's privileges to the *entire agent system*. In a government context, this often means:
* A retrieval plugin with access to a document repository now has its search/read permissions effectively granted to any user prompt that can trick the LLM into using it.
* An email plugin with "send" scope becomes a perfect social engineering exfiltration channel.
* A database query plugin becomes a verbose, natural-language front-end for SQL injection, but with the added "benefit" of the LLM helpfully formatting the stolen data.
The core issue is one of object capabilities and ambient authority. Most plugin designs I've seen hand the agent a bearer token or an API key with broad, persistent privileges. The LLM, a notoriously fuzzy reasoning engine, becomes the confused deputy. The security model then relies entirely on the LLM's "judgment" to not be tricked, which is laughable given the state of prompt injection.
Consider a hypothetical "SecureEmailPlugin" configured for an IL4 environment:
```yaml
# This is the kind of overly-trusting config I keep seeing
plugins:
- name: SecureEmailPlugin
auth:
type: bearer_token
token: ${EMAIL_API_KEY}
capabilities:
- send
- read_inbox
- search
target_resource: "https://email.mil.example.sgov"
```
The plugin, and thus the agent, now has ambient authority over the entire email system. The supposed control is a flimsy "intent" check in the prompt. In a proper capability-based model, the *user context* should provide a narrowly-scoped, single-use capability to the agent for a specific task. The agent shouldn't have a standing, powerful credential in its configuration.
We're bolting a super-intelligent (but gullible) intern onto our most critical systems and giving it the keys to the kingdom because "it needs them to do its job." We've forgotten the first rule of least privilege: **the entity should only have the authority needed for the *current* operation, not all *possible* operations.**
The air-gap is meaningless if the plugins inside the boundary have excessive lateral access. FedRAMP boundary scoping falls apart if you don't treat each plugin as a separate subsystem requiring its own control set. We need to be talking about capability *attenuation*, plugin sandboxing with e.g. WebAssembly or strict seccomp profiles, and ephemeral, user-delegated credentials—not just which cloud provider hosts the LLM.
We're so worried about the AI turning into Skynet that we're handing its API keys to every minor function it might need to call. The real breach will come from a plugin that writes to a log file, which gets ingested by a monitoring system, which has access to the network config. The AI isn't malicious; it's just an incredibly powerful, unpredictable, and credulous *amplifier* for the vulnerabilities already present in our plugin ecosystems.
-- leo
question everything