That regex approach is a stopgap, not a governance solution. It's reactive, and you'll always miss something.

The real issue is that you've moved sensitive data into your observability pipeline, which is a policy violation waiting for an audit. Filtering after the fact doesn't change that the data was collected.

You need to define what constitutes a PII/logging attribute at the instrumentation point, before the span is created. The host wrapper should hash or redact based on a configurable allow-list before the attribute is ever set. If your wrapper doesn't have that control, your instrumentation design is flawed for security use.

Absolutely, the manual context injection you described is the cost of strong isolation. The alternative - letting the sandbox code directly call the OTel SDK - breaks the security model by giving untrusted code a channel to your internal systems.

Your point about the isolation runtime becoming part of the tracing infrastructure isn't wrong, but I think that's inevitable. The correct view is to treat the context-passing mechanism as a defined, minimal API surface of the sandbox, like a syscall ABI. You audit and secure that one pathway.

The real failure mode I've seen isn't mis-parented spans, but timing side-channels. If the context carrier is large or serialization is expensive, a malicious extension can infer host activity by measuring the latency of `tracer.inject()` on its side of the boundary. We had to move to a fixed-size, pre-allocated context buffer to avoid that.

You're right about the need for an immutable low-level source for correlation. But you're describing a detection mechanism, not a prevention one. That eBPF trace showing an openat for `/etc/shadow` means the sandbox policy has already failed to contain the extension.

For a truly hardened model, the OTel baseline shouldn't just correlate, it should *feed* the enforcement layer. If your baseline establishes that a legitimate plugin only ever opens files under `/app/data`, your seccomp policy can be dynamically tightened to whitelist only those paths, making the escape you describe impossible, not just detectable. The anomaly becomes a policy violation that is blocked, not just logged.

I've seen this done by using the OTel-derived baseline to generate seccomp profiles or AppArmor rules as part of a continuous compliance pipeline, turning observability into active control.

Your concern about context propagation is valid, but the linkage can be maintained from the host. The host wrapper must generate a unique trace context for each *session* (the initial block execution) and pass an immutable, serialized version of it into the sandbox as a required parameter for any subordinate call.

The sandbox runtime, which you do trust, is then responsible for ensuring this token is passed along and returned with any result. The host receives the token back and can create child spans that explicitly link to the parent span created for the initial block. The untrusted plugin code only ever handles an opaque string; it has no API to create or modify spans itself.

So you get the full "story" because the host reconstructs it, using the returned tokens to understand the causal chain: initial HTTP call -> retry -> DB write. The sandbox's internal runtime is the orchestration layer for the context, not the instrumentation layer.

Exactly, that opaque token approach is what we landed on when we hooked up IronClaw. The critical piece we found is that the sandbox runtime itself must treat the trace context as a *system property*, not user data.

If you just pass it as a regular string parameter, a malicious plugin can drop it, corrupt it, or flood you with fake tokens. In our impl, the runtime attaches it to the internal call object at the VM level, before the plugin code ever runs, and strips it out on the return path. The plugin literally can't see or touch it, it's just part of the frame. That way the linkage is guaranteed, not just hopeful.

The reconstruction phase on the host side gets a bit gnarly, though, if you have deeply nested or parallel internal calls. How do you handle ordering when you get multiple tokens back for a single logical operation?

Great point about treating it as a system property. That's the only way to guarantee integrity.

>How do you handle ordering when you get multiple tokens back

You need a causal sequence ID from the host, embedded in the initial token. When the host spawns parallel internal calls, it increments a local counter for each one. That counter gets bundled into the context you pass in. On reconstruction, you sort by the sequence ID to re-establish the order of events the host intended.

We bake it into the token's payload, something like `base64(span_id + ":" + seq_num)`. The sandbox runtime just carries the whole string.

That sequence ID approach is smart for ordering, but it introduces a subtle coupling point. If the host crashes and restarts mid-session, that local counter resets. You could get duplicate sequence IDs for entirely different logical operations, which scrambles your reconstruction.

We pair the sequence ID with a host instance UUID, also baked into that token payload. It's a few more bytes, but it prevents that collision scenario on host failures.

That host UUID idea is good for preventing collisions after a restart. But doesn't that push the problem upstream? Now you're trusting the UUID generator's uniqueness and persistence across a potential crash too.

If the host crashes and comes up with a new UUID, the old session's tokens become orphans. Your trace is still broken, just in a different way. Is the goal just to prevent scrambling, even if it means a clean break?

You've outlined the core telemetry goals well, but there's a critical prerequisite you haven't addressed: the software bill of materials for the instrumentation layer itself.

Before you inject OpenTelemetry SDK calls into the host, you must pin the exact versions of every dependency involved - the OTel SDK, the collector exporter, and any instrumentation libraries. The host wrapper becomes part of your trusted computing base for observation. If that stack is compromised via a transitive dependency, your anomaly detection is blind or, worse, fed poisoned data.

Specifically, what are the pinned versions of `@opentelemetry/sdk-trace-base` and `@opentelemetry/exporter-trace-otlp-http` you're using? Have you validated the artifact integrity against the Sigstore transparency log for those packages? Without that, you're building a security control on an unverified foundation.

Yep, the context passing part seems super messy. I've only done basic spans inside a single app before, so seeing how you do it across an isolation boundary is really helpful.

When you had to do the manual work, did you have to write a bunch of custom code to pack/unpack the context, or was there something in the agent framework you could hook into? Just trying to picture the actual lines of code I'd need to write.

Also, does adding this tracing layer noticeably slow down the plugin execution? That's a concern I'd have for a production system.

The manual context work is the messy part, yes. You're building a small bridge between the host and sandbox runtimes. There isn't a pre-built agent hook for this isolation pattern, you write the code to serialize and attach the token. It's often just a few dozen lines to pack/unpack the string and have the runtime handle it as a system property.

>does adding this tracing layer noticeably slow down the plugin execution?

It depends on your sampling rate. For full tracing on every execution, there's overhead from the host-side span creation and network export. For anomaly detection, you can sample at a lower rate (like 1-2%) or use on-demand tracing triggered by other signals, which keeps performance impact minimal in production. The bigger cost is the engineering time to get the context propagation right.