The core challenge with AI agents in IL4/5 environments isn't just about data exfiltration. It's about proving a negative: that the runtime isn't performing unauthorized model updates or incremental learning from protected information. In a FedRAMP context, "the system" includes the agent, its runtime, and any supporting APIs. If you can't demonstrate control over that learning function, your boundary is broken.
Most vendors hand-wave this with "the model is static," but that's a product claim, not an architectural control. You need evidence built into the deployment.
Key points for assessment:
* **Artifact Integrity:** Can you cryptographically verify the exact model binary deployed in production against the one that completed FedRAMP authorization? This needs to be an automated check, not a PDF report.
* **Runtime Constraints:** The execution environment must enforce write restrictions on the model files and vectors. This goes beyond basic file permissions—think immutable infrastructure patterns or runtime security controls that block memory-persisted updates.
* **Telemetry and Logging:** You need detailed, immutable logs of all inference calls, showing input/output character counts or token usage. A spike in processing for a given input size could indicate something beyond inference. This data must feed into your continuous monitoring.
The compliance burden falls on the agency. If the vendor's solution treats the agent as a black box, you're inheriting an unacceptable risk. The question isn't about promises; it's about what you can actually audit and monitor within your accredited boundary.
Show me the threat model.
So if the model itself is a static file, how do we even check the runtime isn't secretly keeping notes somewhere else? Like, what if it writes learned patterns to a separate log or cache that gets read back in later? Is that covered under "memory-persisted updates"?
Exactly. The artifact integrity check is necessary but insufficient on its own. You need to enforce constraints at the system call layer to make that product claim verifiable. A SHA256 sum on the model file means nothing if the runtime can mmap it, modify pages in memory, and then mprotect it back to read-only, or if it can open a separate file descriptor to a shadow copy for writing.
The control has to be that the agent process, and any child processes, are placed in a seccomp-BPF filter that denies key syscalls like openat, creat, mkdir, link, rename, and mprotect with PROT_WRITE on the relevant memory regions. This is where cgroups v2 and namespace isolation become critical - you must ensure the process can't even reach a writable filesystem or device. Without that, you're still trusting the application's own promised behavior.
This shifts the proof from "we have a hash" to "the kernel prevents the writes that would invalidate the hash."
Least privilege is not optional.