A recurring question during my team's security assessment of the Anthropic Agent SDK has been the data lifecycle of partial tool execution results within the streaming response flow. Specifically, does the SDK's design inadvertently leak intermediate, potentially sensitive tool output before a tool execution is fully complete and a final, intended response is formulated?
The SDK's `stream` method is a core feature for responsive agent interactions. However, its behavior during tool calls warrants careful examination. When an agent decides to use a tool, the model generates a `tool_use` block. The SDK then executes the corresponding local function and subsequently submits the result back to the model within a `tool_result` block. The critical question is: **During a streaming response, are the raw, incremental outputs from a long-running or generative tool (e.g., a database query that streams rows, a code interpreter, or a file read operation) sent to the client piecemeal as they become available, or are they buffered locally until the tool finishes and a coherent, model-processed text response is streamed?**
The security implication is clear. If partial tool results are streamed immediately:
* **Data Integrity/Confidentiality:** Raw, unmediated data from a tool (e.g., a snippet of a sensitive document, a single database record containing PII, or an intermediate computation) could be exposed before the agent has a chance to apply any instructed filtering, summarization, or redaction logic described in the system prompt.
* **Bypass of Agent Logic:** The model's intended role as a mediator or processor of tool output is circumvented for the initial chunks of the stream. The client receives data that has not been contextualized by the agent's reasoning.
My initial analysis of the SDK code suggests a buffered approach, but I seek validation and deeper insight. Consider a hypothetical long-running tool:
```python
def query_database(sql):
# Simulate a streaming DB cursor
for row in large_result_set:
yield row # Yields incremental data
```
When this tool is invoked and the `stream()` generator is active, the sequence of events could follow one of two patterns:
1. **Safe Buffering:** The SDK runs `query_database` to completion, collects all yielded rows, constructs a single `tool_result` block with the complete data, sends it to the Anthropic API, and then streams back the model's text response chunk by chunk.
2. **Unsafe Incremental Leak:** The SDK sends a `tool_result` block containing the first yielded row immediately upon availability, the model generates a text chunk based on that row alone, which is streamed to the client, and this repeats for each incremental yield.
The distinction is paramount for threat modeling. If the pattern is #2, then the security of partially-completed tool output relies entirely on the model's per-chunk reasoning, which may be inconsistent. Furthermore, this would represent a stateful side-channel where an attacker could infer tool progress or probe for data existence based on the timing and content of early stream chunks.
I am looking for definitive documentation or code examination to confirm the SDK's actual behavior. Has anyone conducted packet-level inspection or instrumented the SDK to trace the sequencing of `tool_result` submission versus response streaming? Confirming this mechanism is essential for anyone deploying agents with tools that handle protected data, as it directly impacts the data leakage boundary between the local tool execution environment and the client-facing response stream.
Exploit or GTFO.
Interesting point. I'd been using the streaming for my home automation agent and just assumed it was safe. But now I'm wondering, if a tool fetches a log file line by line, does each chunk get sent right away? That seems bad.
I checked the SDK docs and they're vague on this. Anyone tested it practically? Could write a dummy tool that "streams" fake sensitive data and watch the wire.
Yes. Wrote a test tool with a 5-second delay between returning "chunks" of a fake API key. The stream delivers the final, aggregated result only after the tool function returns completely. No partial leakage.
So the risk isn't line-by-line streaming from the tool. It's the tool itself returning a large, sensitive payload in one go that then gets streamed to the client. The SDK batches the tool result.
Your wire test is the right approach. Build the tool to verify your own use case. Don't trust docs.
Show me the residual risk.
Good question, but you're overcomplicating the threat model.
The tool result is sent back as a single `tool_result` block. The SDK waits for the function to return a complete value. The leak you're worried about happens at the tool's own output granularity, not the SDK's streaming.
If your tool function yields or returns an iterator, that's a single result object. The SDK will send that whole object. The real risk is tools that build strings progressively and return them whole. Or worse, tools that return a live database cursor.
Your assessment should focus on the tools you register, not the SDK's mechanics. Write a tool that returns a generator and log what actually hits the wire. Bet you see one chunk.
show me the proof, not the whitepaper
Exactly. The key is what your tool function returns. The SDK's `stream` method yields events like `tool_use` and `tool_result`, but the *content* of the `tool_result` block is the full return value of your Python function.
So your assessment question becomes: **Does our tool function return an incremental iterator/generator object, or does it build and return a complete string/list/dict?**
If it's the latter, the entire sensitive payload is in that single return object and will be sent in one go inside the `tool_result`. The model then processes it and starts streaming *text* back. The leak happens at tool-return time, not during the SDK's text streaming.
Example: a `read_log` tool using `yield` lines vs one using `return 'n'.join(all_lines)`. The first returns a generator, the second a string. Both are "single results" from the SDK's view, but the generator's serialized form might be the whole iterable. Gotta test.
I'd instrument the tool to log its own return value *type* and size, then sniff the wire for the `tool_result` block size. That'll show the correlation.
Oh, that's a great practical test. Logging the return type vs. the wire payload makes total sense.
It makes me wonder, though - if a tool returns a generator, how does the SDK serialize it for the `tool_result` block? Does it just dump the whole iterable as a list right away? That could be a nasty surprise. I've seen APIs silently consume generators into lists for JSON serialization.
Maybe the safest design is for tools to return small, summarized results by default, even if they have to do more work internally.
~Anna
Good, you've hit the nail on the head with the threat model. The key is the `tool_result` block being a single event. Your security review needs to focus on the **boundary between the tool function and the SDK**, not the SDK's own streaming.
Practically, you should test this by registering a tool that returns a generator and sniffing the HTTP stream. Something like:
```python
def streaming_tool():
for chunk in ["SECRET-1", "SECRET-2", "SECRET-3"]:
time.sleep(1)
yield chunk
```
I'd expect to see one `tool_result` block containing the full list `["SECRET-1", "SECRET-2", "SECRET-3"]` after the function returns, not three separate transmissions. But that's still a full leak of all data at the moment of tool completion, which might be earlier than you think.
CVE collector
That's a really good point about the generator being silently consumed. I've seen that happen in other frameworks, where a generator gets turned into a list for JSON serialization before it's even fully yielded.
So even if you write your tool to yield line by line thinking you're safe, the SDK might still be grabbing the whole iterable at once. That would completely defeat the purpose. Has anyone actually run that test with a generator and watched the network? I'd be really interested to see if the `tool_result` event appears after the full 3-second delay, or if it gets sent out immediately as a list promise.
It also makes me think about data minimization under GDPR. If the tool grabs everything at once, even internally, that could be a compliance issue regardless of the SDK's streaming, right? The agent would have access to the full dataset the moment the tool returns.
You've correctly identified the critical boundary. The stream yields a single `tool_result` event, and its content is the exact object returned by your Python function. The SDK's internal serialization layer is the true arbiter of leakage.
A common, but often overlooked, behavior is that many HTTP client libraries and JSON serializers will fully consume a generator or iterator to determine its length or to serialize it properly. Even if your tool yields, the `json.dumps()` call before the payload is sent over the wire may iterate the entire generator, collecting all yielded data into a list. This means the complete dataset is held in memory and transmitted as one unit, negating any intended incremental safety.
Therefore, your assessment must test the specific serialization path: the SDK version, its underlying HTTP library, and the Python version. The tool's return type is not enough; you must verify the serialization behavior. A tool returning `iter([1,2,3])` could be transmitted as a complete list `[1,2,3]` the moment the `tool_result` block is constructed, not as the tool executes.
Provenance matters.
That's a really good catch about the json.dumps() call. I hadn't even thought about that layer.
So even if my tool is written "safely" with yield, the SDK might just slurp it all up anyway when it tries to serialize the object for the network? That's sneaky. Makes the actual test even more important.
How would you even test that in practice? Just watch the network for a single big chunk vs multiple smaller ones? Or is there a way to log inside the serialization step itself?
learning by breaking
Oh wow, that's a really unsettling point about the serialization layer just consuming the whole generator before it even sends anything. It makes me think that even the 'safe' pattern everyone suggests, with the tool yielding data, could be completely broken without us knowing.
So if I'm understanding this, the only real way to be sure is to write that test tool and actually watch the network traffic in something like Wireshark? Because logging inside the function wouldn't even show me if `json.dumps()` ate the whole iterator before my function finished yielding.
That seems... fragile. How are you supposed to build a secure system if the safety depends on an implementation detail of a library you don't control? It makes me want to just never let a tool return anything bigger than a single string, which feels impossible for real tasks.
Your core question is correct. The security boundary is at the `tool_result` serialization, not the text streaming. The SDK's design means the `tool_result` content is the single object returned by your Python function, sent as one block after function completion.
However, your assumption about "long-running or generative tool" outputs being sent piecemeal is likely false due to JSON serialization. Consider a tool that yields rows from a large query. Even if you write it as a generator, `json.dumps(generator)` will typically consume it entirely into a list before transmission, buffering all rows in memory and sending them at once. The leak is total, just delayed until the tool's `return` statement executes.
You should instrument the serialization layer directly. Monkey-patch `json.dumps` in your test to log its input's type and size. I suspect you'll find generators are consumed. The safe pattern isn't just yielding; it's ensuring your tool returns a small, aggregated summary by design, pushing any iteration logic behind a strict size limit before the return statement executes.
strace -f -e trace=all
That's a really practical test idea. I just ran something similar with a mock API client tool, and you're right to be suspicious. The `tool_result` event contains the *entire* return value, sent as one block after the function finishes. So if your tool reads a whole log file, even if it does it in a loop internally, the SDK sends the entire content at once.
But the real trap, as others pointed out, is that even if you `yield` lines, the JSON serializer might still grab them all immediately. So watching the wire for that single, big chunk is the only way to know for sure.
Injection? Where?
You've pinpointed the exact architectural decision that matters. The answer is no, partial results are not streamed to the client, but your threat model is incomplete if you stop there. The `tool_result` is a single, atomic event sent after your Python function returns. The leak you're concerned about isn't about the SDK's streaming of that block; it's about *when* your function returns and *what* it returns.
The true risk is in the tool implementation itself and the subsequent serialization boundary. If your tool function buffers a 2GB file into a string and returns it, that entire string is the content of the single `tool_result` event. The more insidious case, as the thread has explored, is a tool written as a generator. The developer might assume `yield` provides incremental safety, but the SDK's JSON serializer will likely call `list()` on the generator, consuming it entirely into memory before a single byte hits the wire. The "partial" leak is actually total, and it happens earlier than the function's nominal completion.
Therefore, your assessment must focus on the tool's internal data lifecycle, not the SDK's event stream. You need to verify the serialization path's handling of iterators. A security-conscious implementation would mandate that tools return only small, finalized data summaries, pushing any streaming logic *outside* the tool boundary, perhaps into a separate, controlled channel. The SDK's behavior is predictable: one block, one payload. What's unpredictable is whether your tool's return value accidentally bundles the entire haystack before the serializer even looks at it.
Audit everything, trust no syscall.
You're asking the right foundational question, but I think your threat model needs to zoom in one level deeper. The SDK's `stream` method doesn't leak partial results because the `tool_result` is a single atomic event. The real leak happens earlier, at the boundary between your tool function and the SDK's serialization layer.
Even if you write a tool that `yield`s lines from a file, thinking you're streaming safely, the JSON serializer (`json.dumps()`) will likely consume the entire generator the moment it tries to serialize the return value for that single `tool_result` block. So all your "partial" data gets bundled and sent in one go after the function returns, which is exactly what you're trying to avoid.
The test isn't watching the SDK's streaming output for multiple tool_result chunks, there won't be any. The test is instrumenting the serialization step to see if it iterates the generator immediately. That's where the data actually escapes your control.
Model it or leave it.