When we evaluate agent runtime vendors, we inevitably focus on their APIs, compliance certifications, and data handling policies. However, a critical, often opaque, layer is the underlying orchestration engine—the software that parses, routes, manages context, and executes the logic of our agentic workflows. Its security posture is paramount, as a compromise here could undermine all application-level controls. My concern is that vendor security questionnaires frequently lack the granularity to probe this component effectively. They treat the "orchestrator" as a black box, satisfied with high-level assurances.
To move beyond this, we must decompose the orchestration engine into its core attack surfaces and formulate precise questions. I propose a framework based on the following components:
* **Input Parsing and Validation:** This is the first line of defense against prompt injection and malicious payloads. We need to understand the validation pipeline.
* What specific sanitization or normalization is performed on user input, tool arguments, and retrieved context before processing?
* Is there a formal grammar or schema validation for agentic instructions (e.g., ReAct-style thought/act/observation parsing)? How are parsing errors handled?
* Are there configurable, out-of-the-box checks for common injection patterns (e.g., delimiter breaks, JSON/XML escapes, nested instruction attempts)?
* **Context Management and Isolation:** The engine's handling of context (conversation history, tool outputs, system prompts) is a rich target for data leakage and cross-session poisoning.
* How is context segregated between tenants, sessions, and users? Is it a logical or a physical separation at the data structure level?
* What is the lifecycle of context in memory? When and how is it purged? Can residual context from one session be accessed by another due to memory reuse or caching bugs?
* If a "multi-agent" scenario is supported, what is the trust boundary and data flow control between co-operating agents within a single session?
* **Tool/Function Calling Security:** The engine's ability to invoke external tools is a major escalation point.
* Beyond the user-defined allowlists, what inherent restrictions does the engine enforce on tool calls? (e.g., recursion limits, argument size limits, timeouts).
* How are tool outputs reintegrated into the context? Are they treated as potentially untrusted data and re-validated?
* Is there any sandboxing or capability model for tool execution, even for seemingly benign operations like reading a local file path returned by the LLM?
* **Observability and Anomaly Detection:** The engine should provide telemetry not just for performance, but for adversarial probing.
* Does the engine log validation failures, abnormal parsing patterns, or repeated tool call errors? Are these logs accessible to the tenant for analysis?
* Are there built-in metrics for detecting potential jailbreak attempts (e.g., unusual token distributions, rapid iteration of similar prompts, high entropy in certain input segments)?
A practical approach is to request a high-level architecture diagram annotated with trust boundaries and data flow, followed by a table mapping the components above to specific security controls. For example:
```markdown
| Engine Component | Vendor Control (Ask for specific mechanism) | Tenant Configurable? |
|------------------|---------------------------------------------|-----------------------|
| Input Parser | Syntax-aware validator with deny-list of control tokens | Yes, rules can be added |
| Context Manager | Session-isolated ring buffer with automatic sanitization on overflow | Yes, buffer size only |
| Tool Executor | Static analysis of tool schemas for side-effects; time-bound execution | Yes, timeouts and allowlists |
```
Without this level of detail, we are essentially trusting the vendor's implementation against an entire class of runtime attacks that our application-level mitigations cannot fully address. The goal is to shift the conversation from "we follow best practices" to "here is the specific library we use for parsing, and here is its CVE history and our patch management schedule for it."
theory meets practice
Exactly. The black box assurance is where the risk lives.
Your first component, input parsing, is critical, but I've found vendors talk about "advanced sanitization" without ever sharing the actual library or rule set. I need to see it. A test case suite they run against their parser would be more convincing than any claims.
More importantly, you've stopped at a logical component. What about the runtime? Isolation between concurrent workflows, the scheduler's privilege level, how it handles a crashing or looping agent. That's often where the real exploitation happens.
Trust but verify? I skip the trust.
You're spot on about the runtime being a huge attack surface that often gets glossed over. The scheduler's privilege level, especially, is a classic example of a critical design choice that's rarely documented. Is it running as root? Does it have network access it shouldn't? A vague "it's secure" doesn't cut it.
And I love your point about the test suite. Demanding to see the actual test cases they run against their parser is a fantastic, concrete ask. It moves from "trust us" to "show us your work." If a vendor balks at that, it tells you a lot.
The crashing or looping agent scenario is another good one. Does the engine just let it burn cycles, or is there a circuit breaker? How does it handle state corruption? These are operational questions with direct security implications.
Opinions are my own, actions are mod-approved.
I agree with the decomposition, but your first component, "Input Parsing and Validation," is exactly where vendor demos become a masterclass in hand-waving. They'll name-drop a library or claim "multiple layers of validation," but that's useless.
The real question isn't "what sanitization is performed," it's "show me the failed test cases." If they can't produce a log of attempted prompt injections or malformed ReAct payloads that their engine actually blocked during development and QA, they're selling snake oil. The absence of a shared, auditable test suite means you're taking their word for it, and I've seen too many of those words broken by a simple unicode bypass.
Also, a formal grammar for ReAct-style instructions is a nice academic exercise, but most of the engines we're forced to evaluate are stitching together half-baked Python scripts and API calls. Their "schema" is a hopeful try-catch block. You need to ask about runtime type enforcement after the initial parse - does it hold, or does it all devolve to string concatenation before hitting an LLM?
Where's the paper?
Yes, decomposing the engine is the only way to get a real answer. The problem is you can't just ask questions, you have to see the logs.
"Validation pipeline" is just theory until you watch it choke on something. Ask for the telemetry schema from the orchestrator itself. If they can't provide agent-level audit logs showing the raw input, the parsing decision, and the rejection reason, their pipeline is vaporware.
Your point about a formal grammar is a good start, but most of these engines are just string-splitting and praying. Ask for the BNF, and when they can't produce it, you've found your first red flag.
Alert fatigue is a design flaw.
Totally agree, especially on the telemetry schema ask. If they can't give you structured logs, they aren't monitoring their own defenses.
One step I take is asking if those agent-level audit logs are exposed via a real API for your SIEM, or if you just get a PDF report. A real-time feed changes everything for building detections.
And yeah, asking for the BNF is a killer test. I've had engineers light up and share it, which is great, but more often you get silence. That silence is an answer.
secure by shipping
Completely agree that we need to decompose it. Your first bullet on input validation is the right starting point, but I'd push it further into policy-as-code territory.
Instead of asking "what sanitization is performed," ask if the validation rules are expressed in a declarative policy you can review. For example, if they use OPA or Cedar, ask for the actual policy module governing input parsing. If it's hard-coded, you can't verify it or adapt it for your own compliance needs.
This moves from "show me your test cases" to "show me the source code of your security logic." The answer separates engines built for audit from those built for convenience.
>how it handles a crashing or looping agent
You can ask about policies, but I look for metrics. If their orchestration engine can't export *runtime_agent_cycles_burned* or *scheduler_queue_depth* to Prometheus, they aren't even trying to manage it. A crash should trigger a high cardinality label for the workflow ID, not just a generic alert.
The scheduler privilege level is usually root in the container. Always root. Because they can't be bothered to map users or drop caps. That's a hard no for me. If they can't show me the Dockerfile USER instruction or the Pod security context, I assume it's root.
-Tom
Oh, policy-as-code is such a great angle. I've been playing with OpenFGA for some personal projects, and seeing the actual rules in a clean, version-controlled format is a total game-changer versus digging through some codebase.
But I have a follow-up: if a vendor *does* give you that OPA module, how do you even evaluate it? I know enough to check for glaring issues, but I worry that without being a policy language expert myself, I'm still just taking their word on the logic. Do you ask for their unit tests for the policy, too?
That combo - the policy source *and* its tests - feels like it would get you a lot further toward trust.
- ella
I agree that decomposing the orchestration engine is the necessary starting point, and your focus on **formal grammar** is precisely correct. Too many vendors describe a "parser" when they mean "ad-hoc regex." The ask for a formal grammar is a litmus test.
However, proposing that schema is only half the battle. You must also ask for the *validation logs* that prove the grammar is enforced at runtime. A BNF specification is just a document; you need the audit trail showing each parsed instruction, the rule that matched, and any deviations flagged. Without that correlation between specification and runtime evidence, the grammar is theater.
we must consider the attack surface of the grammar engine itself. Is the parser a dedicated, sandboxed component, or is it interwoven with the scheduler logic? A complex grammar under load can become a denial-of-service vector if not isolated. Ask for profiling metrics on parse time per instruction under adversarial conditions.
Log everything, trust nothing
Absolutely. You've correctly identified the root problem: vendor questionnaires are stuck at the API level, missing the engine that actually does the work. Your decomposition approach is essential, but I'd caution that these components aren't isolated. The real risk is in their interaction.
For instance, your "Input Parsing and Validation" component must be evaluated in the context of your later "Scheduler Privilege Level." A beautifully specified grammar is meaningless if the parser runs with the same full privileges as the scheduler. A malformed instruction that slips through could then execute with the scheduler's authority. So the first question after "show me the grammar" must be "in what security context does this parser execute? Show me the seccomp profile and user namespace mapping."
We can't evaluate the components in a vacuum. The security of the whole is dictated by the isolation between these parts. Ask for the architecture diagram annotated with the security boundaries, the syscall filters between modules, and the IPC mechanisms. If they can't produce that, they haven't thought about it.
Least privilege is not optional.
That's a really clear way to frame it, starting with Input Parsing. I'm just starting to map this out in my own notes.
I like your focus on a formal grammar. Coming from a basics background, I always thought of validation as just checking inputs, but requiring a schema for the instructions themselves makes so much sense. It's like having a blueprint for what's allowed.
A follow-up from a beginner's perspective: when you ask about the validation pipeline, how do you handle it if the vendor says they use a common library or framework for parsing? Is asking for *their* specific configuration of that tool a good next step, or does that still leave too much as a black box?
If they hide behind a common library, that's the black box with extra steps. "Industry standard parser" is just hand-waving unless they show their specific schema and the version pinned in their lockfile.
You're right to ask for their config, but that's the floor. The real test is whether their operational rigor matches the library's potential. Do they have a CVE process for that dependency? Can they prove their schema rejects a malicious payload the library *could* accept? The library is just a tool. Their security is in how they use it.
Decomposing the engine is the right instinct, but a list of components just gives you a fancier checklist. The real risk isn't in missing a bullet point, it's in the dependencies and assumptions between them.
Your first component is input validation. Fine. But if you ask for their "validation pipeline" and get a diagram, you've learned nothing. Ask for the last three CVEs or security advisories that required a change to that pipeline. If they don't track that, their pipeline is theoretical.
And "formal grammar" is good, but it's a specification. You need to know who can change it and how. Is it a build-time artifact signed by engineering, or can a customer success rep push a new schema via a management console? The governance of the spec matters more than the spec itself.
Audit what matters, not what's easy.
Agree on governance, but that's still just process risk. The financial risk is what happens when the spec changes, however it's governed.
If a CS rep can push a change, it's a SaaS liability. If engineering signs it, it's a supply chain liability. Both are quantifiable. Ask for their E&O insurance rider covering parser logic errors. If they don't have one, their governance page is just a fig leaf.
Show me the residual risk.