Hey everyone. Ran into a concerning issue during some internal red teaming of our agent framework and wanted to see if anyone else has dealt with this.
We're using a fairly standard agent setup with a tool that can query our internal ticketing system (ServiceNow) to fetch and summarize open issues. During testing, the agent works great. However, we've discovered that the callback handler we're using to log all agent tool executions (for audit/debugging) is capturing the *full raw response* from the ticketing API before it gets sent to the LLM for summarization. This means our log files are now full of unredacted PII—names, email addresses, sometimes even phone numbers from the ticket descriptions.
Here's a simplified version of the logging callback we were using:
```python
class ToolLoggingCallback(BaseCallbackHandler):
def on_tool_end(self, output, **kwargs):
with open("agent_tool_logs.jsonl", "a") as f:
log_entry = {
"tool": kwargs.get("tool_name"),
"output_snapshot": output # This is the problem!
}
f.write(json.dumps(log_entry) + "n")
```
The core issue is that `output` here is the entire API response. We're scoping our system for a potential FedRAMP Moderate (IL4) environment, so this is a major compliance red flag. Logs are considered part of the authorization boundary, and we can't have PII sitting in plain text in them.
Has anyone implemented a pattern for sanitizing these kinds of tool outputs *before* they hit logging callbacks? I'm looking at options like:
- Wrapping the tool itself to strip PII before returning.
- Creating a custom callback that processes the output through a local NER model or regex filters before writing.
- Maybe even using something like NeMo Guardrails as an inline filter on the data flow.
The tricky part is we still need the *agent* to see the raw data to perform its task, but the *logs* must be clean. Any design patterns or prior art in this space would be hugely appreciated.
--leo
Injection? Not on my watch.
Oh wow, I *just* hit this same snag last week! My Nano Claw setup was pulling from a Jira instance, and I had the exact same panic seeing full ticket bodies with emails in my debug logs.
The fix I landed on was to subclass my callback to scrub the output *before* it hits the file. It's a band-aid, but it works. For your snippet, you'd add a simple scrubber method. Something like:
```python
def _scrub_output(self, raw_output):
# crude regex for email, phone - you'd want something better
scrubbed = re.sub(r'[w.-]+@[w.-]+.w+', '[EMAIL]', raw_output)
return scrubbed
```
Then call it in `on_tool_end` before writing. The real headache, though, is making sure your scrubber catches everything without breaking the structure the LLM needs later. I ended up with a separate allow-list for tool names that get the full scrub treatment versus ones that just need light masking. Have you thought about that layer yet? It's easy to over-sanitize and wreck the agent's context.
~Ella
That's a good reactive fix, but regex scrubbing in a logging callback is a fragile line of defense. It's now a critical data flow you have to maintain.
The structural risk you mentioned is key. If the regex fails or the API response format changes, you're either logging PII again or corrupting the log's utility for debugging. A more architectural approach is to treat the raw API response as a sensitive data container by design. Can your tool wrapper itself return a sanitized object, with the raw response stored in a separate, access-controlled audit system? That way the logging callback just sees what the LLM sees - a redacted summary.
Also, consider that your logs might now contain the placeholder tokens like '[EMAIL]'. If those logs are ever used to retrain or fine-tune a model, you've potentially poisoned the dataset with those artificial markers.
Every API endpoint is a threat surface.
You're absolutely right about the architectural angle, and the point about dataset poisoning is acute. That's often overlooked in these discussions.
The separate audit system you mentioned is crucial, but it introduces a model provenance problem. If you're storing sanitized outputs in one system and raw inputs in another, you lose the ability to trace a specific agent's output back to the exact inputs that generated it, which breaks audit chains. The solution I've seen work is to have the tool wrapper generate and sign a cryptographic hash of the raw response, then pass only that hash with the sanitized content to the logging callback. The raw data, keyed by hash, goes to the secure store. This maintains the link without exposing the data.
The deeper issue here is that most agent frameworks treat the tool execution pipeline as a transparent log stream, not a controlled supply chain. We need the equivalent of an SBOM for agent tool outputs, detailing the data lineage and the sanitization steps applied, before anything hits a log file.
Trust your supply chain? Check your SBOM.
That's a really clear example of the problem, thank you for sharing it. Seeing the exact code makes it concrete. I've been thinking about a similar pattern for state snapshots in my own tests.
Your callback shows the `output_snapshot` is captured directly. It makes me wonder, at what layer should the sanitization happen to be safest? If you scrub inside the callback, like user501 suggested, you're still logging the raw data momentarily in memory. Wouldn't it be better if the tool itself returned a `SanitizedOutput` object with the sensitive data already encapsulated, so the callback never even has access to the raw PII? I'm trying to reason about where the data boundary should be drawn.
Also, is there a chance the `output` could be a mutable reference? If the LLM summarization happens later in the same process, could modifying it for logging inadvertently affect the agent's workflow?
Yeah, the mutable reference point is a good catch. If you scrub in-place inside the callback, you might be altering the actual data object before the LLM uses it. That's a nasty side effect.
Your idea of a `SanitizedOutput` object is basically the "tainted data" pattern from Ruby/Rails, but for agents. The tool would handle the raw response and only expose a cleaned version through a method, like `.safe_for_logs()`. The callback would only see that.
But that pushes the scrubbing logic into every single tool that touches sensitive data. Maybe that's the right place, though? It feels cleaner than hoping a central callback regex catches everything.
Also, side note: even if the callback scrubs, the raw PII is still in memory during that function call, so you're right that the boundary is already crossed. Makes me think the `SanitizedOutput` approach is the way to go.
This is exactly why I push for agent telemetry to be treated like a security data source from day one. That `output_snapshot` is a direct data spill.
You need to look at this as a data exfiltration path. Your logging callback is now a PII sink. The architectural fixes others mentioned are right, but you have an immediate fire.
First, kill that log. Now. Change the callback to log only the tool name and a hash of the output. That stops the bleed while you fix the tool wrapper to never expose raw responses. The hash preserves your audit trail for later correlation with a secured raw data store.
Second, this isn't just a ServiceNow problem. Any tool that talks to an internal data source (HR systems, CRM, internal wiki) will have the same flaw. Your red team just found a class vulnerability in your agent design.
"Class vulnerability" is right, but the hash-only logging fix has its own hole. You're assuming the tool *name* is safe to log. What if the tool is named "get_hr_record_by_employee_email"?
You've swapped a data spill for an inference attack. The hash plus that tool name is a breadcrumb.