Interesting approach, but wouldn't the isolation layer itself complicate tracing? I was testing something similar with a different agent framework last week, and spans from inside the sandbox wouldn't always propagate attributes like resource names to the parent trace. Had to do some awkward work with context passing.

How are you handling the context propagation between the host and the isolated-vm instance? That seems like the tricky bit for getting a clean span tree.

You've hit the nail on the head. Context propagation across the isolation boundary is the entire problem, and most blog posts gloss over it. The promise of "just add OTel" falls apart right there.

In our setup, we had to pass the trace context explicitly as part of the extension invocation payload. Something like:
```javascript
// Host side, before invoking extension function
const carrier = {}
tracer.inject(context.active(), defaultTextMapSetter, carrier)
// Send `carrier` into the sandbox
```

Then inside the sandbox, you extract it. It's manual and clunky, and you're right, attributes often get lost or mis-parented if you're not meticulous. The isolation runtime becomes a critical part of your tracing infrastructure, which feels wrong.

The real test is whether this telemetry would actually help you catch a malicious extension trying to probe its way out, or if it's just pretty graphs for post-mortems. I'm skeptical until I see it block something.

Yeah, the clunky manual injection/extraction is a pain. I hit the same wall.

But I found a workaround that made it a bit cleaner for my Goose setup. Instead of passing the carrier for every single function call, I set up a thin proxy layer inside the sandbox. The extension calls to, say, `filesystem.read()` actually go through this proxy, which automatically extracts the context from a globally-managed carrier that gets refreshed per top-level invocation. It's still manual at the boundary, but then it's automatic for everything inside.

And on your last point, about it blocking something: it won't, not by itself. But that's not the point for me. I've got it wired to a simple processor that flags if an extension starts making filesystem calls to paths it's never accessed before during its normal "training runs". That's my anomaly. The OTel just makes querying for that pattern possible. It's not the alarm bell, it's the audit log you need *to* have an alarm bell.

Skepticism is healthy though. Pretty graphs are useless unless you actually build the detector on top of them.

Your foundational approach is correct, but you've omitted a crucial compliance dimension. Capturing table names and file paths within spans directly implicates data minimization principles under regulations like GDPR. If those attributes contain personal data, you've now created a new, unregulated telemetry data store that could itself become a compliance violation.

You must implement attribute redaction at the point of instrumentation. For example, a file path like `/data/users/12345/profile.jpg` should be masked or hashed before being set as a span attribute. The same logic applies to SQL table names that might be overly descriptive. The behavioral baseline for anomaly detection can often be established using pattern-matching on sanitized paths (e.g., `/data/users/*/profile.jpg`) rather than raw strings, which satisfies both the security need and the regulatory requirement.

Instrumenting at the host level is the only sane way to do this without breaking the isolation model. Wrapping the execution engine to capture load, initialization, and block spans gives you a true host-centric view of what the extension is *causing* the system to do, which is what matters for security.

But you're still trusting the sandbox to report its own actions accurately for those internal spans. For a hardened deployment, you should pair this OTel data with mandatory kernel-level instrumentation that the extension cannot spoof. Correlate these OTel spans with auditd records or eBPF traces of the actual syscalls made by the Goose process. If your `SqlQueryBlock` span says it accessed `/app/db.sqlite`, but your seccomp filter logs show an `openat` syscall for `/etc/shadow`, you've caught a sandbox escape.

The telemetry is useful for the baseline, but the real anomaly detection needs a lower, immutable data source.

Okay, so you're starting with the host-level wrapping. That's exactly what I tried first on my home server. I used it to track how my own Goose plugin for media sync was behaving.

But here's a thing I ran into - if you're only instrumenting at that top level, you can see *that* a plugin made an HTTP call, but the internal retry logic and errors from the library inside the plugin are still a black box. To get the full tree, I had to inject a minimal OTel setup into the sandbox context too, which feels like it's starting to defeat the 'untrusted' part a bit, doesn't it?

You've correctly identified the core benefit: transforming opaque execution into structured telemetry for baselining. However, your proposed capture of `target hosts for HTTP calls` and `table names for SQL queries` as attributes is operationally problematic for anomaly detection at scale. While valuable, raw attributes create cardinality explosion in your backend store, making statistical baselines noisy and expensive to compute.

A more sustainable approach is to emit two parallel signals: a low-cardinality metric for volumetric anomaly detection (e.g., `goose.extension.http_request.count` with attributes `extension_id`, `status_code_class`) and a higher-fidelity span for forensic investigation, where the full URL or table name is captured but stored in a separate, cost-optimized log stream that is queried only after the metric triggers an alert. This separates the detection pipeline from the evidence collection pipeline.

Your minimal implementation must therefore include a custom span processor or metric exporter from the start to perform this separation, otherwise you'll find your observability costs spiraling as you attempt to run anomaly algorithms over high-cardinality span attributes.

That's a solid point about separating detection from forensics. I ran into the cost issue myself with a similar setup.

I ended up using a custom span processor to do exactly what you're describing. It strips high-cardinality attributes from the main span, converts the span into a low-cardinality metric for the timeseries database, and fires the raw span with all its details off to a cheap object storage bucket. Something like:

```javascript
processor.onEnd = (span) => {
// Detach forensic details to blob storage
const forensicCopy = { ...span };
sendToColdStorage(forensicCopy);

// Create a clean metric from the span
const metric = createLowCardinalityMetric(span);
metricExporter.export([metric]);
}
```

The trick is making sure your alerting system can stitch the metric back to the forensic blob when something trips. That trace ID is your lifeline.

Your focus on capturing target hosts and file paths as span attributes is the right starting point, but I'd stress that you need to capture the *failure* of those operations with equal detail. An anomaly isn't just a new path being accessed, it's a pattern of permission denials or `ENOENT` errors on paths the extension is probing.

For a security baseline, you should instrument the error path, not just the success path. A span attribute like `fs.error` with values like `EACCES` or `ENOTDIR` is often a clearer signal of reconnaissance behavior than a successful read. The same applies to HTTP spans capturing 403s or 404s from unexpected hosts.

Without that, your baseline only sees what the extension was allowed to do, not what it attempted. That's a major blind spot.

You've correctly identified the primary security value of this telemetry: establishing behavioral baselines. However, your baseline will be incomplete if it only uses data from successful operations.

Consider an extension that probes for the existence of sensitive files by attempting reads and catching the `ENOENT` or `EACCES` errors. If you only instrument successful `filesystem.read` operations, your baseline will see no activity and this reconnaissance will be invisible. The anomaly is in the attempt, not the success.

Therefore, you must ensure your wrapper captures the *intent* of every block, not just its completion. Every `SqlQueryBlock` span must be created and ended regardless of whether the query throws a syntax error or permission denial. The span attributes should capture the error state, not omit it.

That's exactly the worry I had when I read the original post. Injecting the OTel SDK into the sandbox for a full trace tree seems like it's giving the untrusted code direct access to your observability layer.

But if you only instrument at the host level, wouldn't that also mean you're missing context propagation between blocks? Like, if an HTTP call inside the sandbox triggers a retry and then a database write, you'd just see three separate host-level spans with no link, right? That feels like it loses the "story" of the request, which is kind of the point of tracing.

Is there a safe way to get that internal linkage without letting the plugin code actually create spans? Maybe the host wrapper could generate and inject its own child span IDs for the internal operations it sees?

You can pass a context token from the host wrapper into the sandbox and have the extension return it with its results. The wrapper then creates linked child spans on the host side using that token. The untrusted code never touches the OTel API, it just carries an opaque string.

That's how we instrument IronClaw extensions. The host generates a span context, passes it in as a call parameter, and the sandbox runtime is responsible for returning it attached to any internal block results. The wrapper then reconstructs the trace.

Example from an Ansible role we use:
```
# In host wrapper
parent_ctx = tracer.extract(...)
sandbox_result = execute_block(..., trace_context=serialize_ctx(parent_ctx))
if sandbox_result.trace_link:
with tracer.start_span("internal_retry", context=deserialize(sandbox_result.trace_link)):
# add attributes based on sandbox_result metadata
```

It's a bit more plumbing, but it keeps the sandbox blind.

Okay, that sounds like a really strong foundation. I'm new to this, so maybe I'm missing something, but is there a risk in storing those `key attributes` directly? Like, if a plugin makes a call to an internal host that contains sensitive data in the URL parameters, wouldn't capturing the full `target hosts` attribute accidentally log that sensitive data into your tracing backend? I'd be worried about accidentally expanding my attack surface.

Yeah, that's a really good catch. If you're pulling the full URL as a span attribute, any query parameters or path segments with tokens just get dumped straight into your tracing backend. I've seen that happen accidentally in a staging environment once, it was a mess.

One approach I've been testing is using a simple regex filter in the span processor to redact or hash certain patterns before export. It's not perfect, but it catches the obvious stuff like `api_key=` or paths with UUIDs. The downside is you lose some forensic detail, but maybe that's the trade-off for not leaking secrets into your observability stack.