Our agency recently concluded a six-month prototype for a retrieval-augmented research agent intended for an air-gapped, IL5 environment. The core challenge was implementing a useful agentic workflow while adhering to strict FedRAMP boundary controls and assuming a complete absence of external APIs post-deployment. This required a fundamental rethinking of component scoping and isolation, moving beyond simple vendor black-box solutions.

The primary architectural decision was to decompose the typical monolithic agent runtime into discrete services, each with a clear FedRAMP boundary classification. We treated the LLM inference itself as a "High" impact system, but the orchestration logic, tool execution, and user interface as separate "Moderate" components. This allowed us to containerize and harden the LLM service independently, applying stricter controls to its direct inputs and outputs.

The prototype stack was built entirely from open-source components to guarantee auditability and ensure no latent external calls. Key elements included:

* **Orchestrator:** A modified version of LangChain's `AgentExecutor`, but with all networking capabilities surgically removed and a custom parser for tool calls that performs rigorous validation against a pre-defined schema.
* **LLM Service:** A vLLM container serving a 7B-parameter model, fine-tuned on internal technical documents. The service endpoint is exposed only to the orchestrator via an internal service mesh, with all inference logs routed to a dedicated SIEM.
* **Tool Isolation:** Each tool (e.g., document search, calculation) runs in its own ephemeral container, spawned by the orchestrator. The tool container receives only the specific, sanitized arguments for its execution context and has no network access other than to required internal data stores.
* **Input Sanitization Layer:** This is the critical control point. All user prompts and retrieved context pass through a series of regex and semantic checks before reaching the orchestrator. The most effective mitigation was implementing a dual-LLM system for classification, though this incurred significant latency.

```python
# Simplified sanitization checkpoint (pre-orchestrator)
def validate_input_for_agent(user_input: str, session_context: dict) -> dict:
"""
Returns sanitized dict or raises InputValidationError.
"""
# Check 1: Deny list for obvious injection patterns
injection_patterns = [r"ignore.*previous", r"system.*prompt", r"..."]
for pattern in injection_patterns:
if re.search(pattern, user_input, re.IGNORECASE):
raise InputValidationError("Pattern match failure.")

# Check 2: Context length bounding
bounded_input = user_input[:MAX_INPUT_LENGTH]

# Check 3: Query intent classification via small, fast model
# This model is loaded separately and only used for this task.
intent = intent_classifier.predict(bounded_input)
if intent not in ALLOWED_INTENTS:
raise InputValidationError(f"Intent '{intent}' not permitted for this session.")

return {"sanitized_input": bounded_input, "intent": intent, "context": session_context}
```

The most significant finding was that prompt injection attempts shift from being a data integrity problem to a potential availability and resource exhaustion threat in an air-gapped system. A successful injection could not exfiltrate data but could potentially cause the agent to enter a loop, spawning thousands of tool containers and degrading the underlying platform. Our mitigation involved strict, stateful rate-limiting at the orchestrator level and circuit breakers on tool calls.

We are now evaluating the trade-offs between this decoupled architecture and a more integrated, but less flexible, single-binary runtime. The main points of contention are the overhead of inter-service communication (which must traverse the internal mesh) versus the security benefits of clear isolation boundaries that align with FedRAMP component definitions. I am particularly interested in forum members' experiences with similar decompositions, especially regarding performance under load and the auditability of the control flow across these boundaries.