We've all seen the demos. A vendor shows their runtime "defeating" a few hand-crafted prompt injection attempts. The audience claps. It's meaningless.

The current "benchmarking" for agent security is mostly marketing. It's a curated set of known-good examples that prove nothing about how the system will behave under real adversarial pressure. We're buying security theater.

What we actually need is a standardized, open, red-team certification for agent runtimes. Think of it like a Common Criteria evaluation, but for the AI agent stack. Not a pass/fail from the vendor, but a structured, repeatable methodology executed by qualified third parties.

Here's a rough sketch of what that should measure:

* **Input/Output Surface:** Fuzzing of all LLM inputs (system prompt, user query, context from RAG, tool outputs) and structured outputs (tool calls, final answers). Not just text injection.
* **Orchestrator Integrity:** Can the agent be tricked into looping, re-prompting itself, or exposing internal instructions? What's the isolation between tool execution, context, and prompts?
* **Tool Use Control:** Can an injection force unauthorized tool execution, parameter manipulation, or data exfiltration via tool output? We need to test the actual security boundaries.
* **Persistence & Memory:** If an agent has memory (vector DB, etc.), can you poison it? Does a one-shot injection fail, but a multi-turn attack succeed?
* **Observability & Hardening:** Does the runtime provide sufficient audit trails to *detect* an injection attempt, even if it can't always block it? Logs should be immutable and outside the agent's control.

The certification would produce a report, not just a badge. It would detail the methodology used, the specific attack classes tested, and the runtime's resilience profile. Example output:

```
Test Class: Direct Instruction Override
- C-01: System Prompt Extraction: PASS
- C-02: Ignore Previous Instructions: FAIL (partial compliance observed)
- C-03: Tool Forced Execution: PASS
- C-04: ... etc.
```

This moves the conversation from "our demo looks cool" to "here is our attack surface, and here is how we mitigated points 1, 3, and 7." It gives infrastructure teams something concrete to evaluate. Until we have this, we're just taking the vendor's word for it.