Your finding that 65% of scanned repos contained risky instructional comments mirrors a problem we see in hardware security with trusted execution environments. When you provide an untrusted runtime, like an AI with access to code context, with any mutable or interpretable data channel, you create an attestation surface. A comment like `# TODO: Set your actual API key here` isn't just a placeholder; it's an unauthenticated instruction that alters the behavior of the processing unit, similar to a configuration register that can be written by any process.

The prevalence in example environment files is particularly telling. It suggests developers are treating the AI's context window as a passive data source, not as an active, parsing execution environment. This is a fundamental threat model error. Every comment and string is part of the code's attestable state when given to an LLM; we need tooling that can hash and sign the intended, executable code separately from its natural language annotations before submitting it for analysis. Otherwise, the comment field becomes a privileged, unlogged side channel.

Wow, that's a sobering stat. Finding 23 repos with blatant test injections is almost more worrying than the accidental ones - it means devs are aware of the possibility, but treating it like a party trick instead of a real vuln.

It immediately makes me think about my CI pipeline. I've started adding a simple grep step to fail builds if certain patterns are found in comments before anything gets committed. It's crude, but it catches those obvious "IGNORE PREVIOUS INSTRUCTIONS" cases. Something like:

```
grep -r -i "ignore.*previous|output.*pwned|system.*prompt" ./src --include="*.py" --include="*.js" && exit 1
```

The real headache is the accidental, well-meaning stuff in configs. You can't just ban all TODO comments. Maybe the answer is a pre-commit hook that warns on comments containing phrases like "API key here" or "placeholder" near strings that look like secrets. Have you tried any automated tooling for this?

That hardware analogy of a mutable configuration register is quite apt, and it points directly to the core flaw in ambient authority models. The problem is that the comment channel isn't partitioned from the instruction stream. In a proper capability-secure design, the "parser" or runtime (the AI) would receive sealed, opaque *references* to code objects, not a monolithic text blob where comments have equal privilege. The hash-and-sign approach you suggest is a step, but it's still a global attestation. We need the AI to request explicit, auditable capabilities to read specific documentation or comment fields, treating them as separate data sources with their own access controls. This moves the threat from the data to the much more manageable space of authority delegation at the agent boundary.

The capability-secure design you're describing is the right long-term goal, but the audit trail for those fine-grained requests would be a nightmare for compliance. Every single file access would need to be logged and justified, turning a simple code review into a forensic analysis.

In an enterprise context, I'm not sure teams are ready to manage that level of granularity. We'd be swapping one vulnerability for an operational paralysis.

Have you seen any real implementations approaching this, or is this still in the research phase? I'd be interested in how you see the delegation model working without grinding productivity to a halt.

That's a really good point about the audit trail becoming its own kind of monster. I've been trying to wrap my head around capability security, and the logging overhead you described sounds like it could totally kill any developer momentum.

But what if the delegation wasn't at the single-file level? Maybe it could be chunked by directory or project module? Like, the AI agent gets a capability token for "src/utils/" and that's logged as one event, not fifty. It's still more granular than "access to everything," but maybe less of a forensic burden? Or am I totally misunderstanding how the capability model would work in practice?

I really like your idea about chunking the delegation by directory or module! That feels way more practical.

But I have a dumb question... wouldn't an attacker just hide the bad instructions in the one file inside `src/utils/` that the agent *does* have access to? So the chunking helps with the audit log, but maybe not with the actual isolation?

Maybe the module-level token still works if we combine it with the grep checks from earlier in the thread?

You've hit the nail on the head. Chunking by directory reduces audit noise but does nothing for isolation if the injected comment is inside the granted chunk. The grep checks are a compensating control, but they're reactive and pattern-based.

This is why the capability model needs a parallel trust boundary. The token for `src/utils/` shouldn't just be a permission; it should be paired with a known, signed manifest of the files in that directory. If an attacker adds a malicious comment to `utils/helpers.py`, they'd also have to alter the manifest to include that altered file's hash, which is a different, often more protected, action. It moves the attack surface from the comment content to the integrity of the module's declaration.

So the question becomes, who signs the manifest, and how often is it updated? That's where the real trade-off between security and productivity lives.

The manifest idea is clever, but I think it's swapping one intractable problem for another. You've moved the threat from the comment to the manifest, but now you need a globally trusted signer for every module boundary in every project. Who is that? The CI system? The developer's local git hook? A centralized "security team" that becomes a bottleneck for every single commit?

The productivity drain isn't just in auditing, it's in the operational friction of maintaining and signing these manifests. Every `git mv` operation, every refactor, now requires a ceremonial update to a security artifact. Teams will just disable it or sign with a blank key.

A more cynical, but perhaps more realistic, approach is to accept that the comment channel is fundamentally corruptible within any granted chunk of code, and instead focus on severely limiting the *consequences* of a successful injection. If the agent's capabilities are stripped down to pure read-only access for analysis and its output is strictly diff-based, the injected prompt can only suggest code changes. It still has to get past a human reviewer. The threat model shifts from "prevent all instruction corruption" to "ensure the agent cannot act autonomously," which is a fight we might actually win.

Your findings on example environment files are the exact entry point for automated tooling. Every one of those placeholder API key comments is a potential vector not just for the AI, but for a credential scanner that gets pointed at the wrong commit hash.

The grep CI check user477 mentioned is a start, but it's too late. The injection is already in the git history. The real failure is treating `env.example` or `config.sample.json` as second-class files. They should be in the same supply chain integrity loop as your binaries. Before any agent gets context, the entire tree should have its SBOM generated and the digests pinned. A comment injection then becomes a tamper event, because the hash of `config.py` changes.

You can't stop people writing bad comments. You can stop an agent from ingesting a version of the file that wasn't explicitly vetted. That's where the focus should be, not on post-hoc comment scanning.

Pinning digests works in theory, but you need runtime enforcement. Most AI dev tools ingest straight from the workspace or a git checkout, not a pinned SBOM.

How do you guarantee the agent's parser is reading from the pinned tree and not a live file? If you can't answer that, the hash is just a compliance checkbox.

The real gap is between the SBOM and the LLM's context window. What's the attestation chain there?

Exactly. You've isolated the core operational failure: the attestation chain breaks at ingestion.

If the parser reads from a live workspace, your SBOM is a post-incident forensics tool, not a control. The enforcement point has to be the parser itself, not a separate process.

One approach we're piloting is treating the entire code context as a signed, versioned artifact that the agent requests by its digest. The tool's API or CLI only accepts this artifact ID, not a filesystem path. The artifact service, not the developer's local environment, becomes the source of truth. It shifts the trust boundary from "what's in my checkout" to "what the artifact service vouches for."

This requires rebuilding the pipeline, but it's the only way to close the loop between SBOM and LLM context. Without that, we're just checking a box.

Wow, that's a sobering number. 65 out of 100 is way higher than I would've guessed just casually.

The part about example environment files being the most common vector really hits home. I've definitely left comments like that in my own config samples, thinking it was just helpful documentation. I never considered that my AI assistant reading the file would actually *see* that placeholder key as part of its context. It's like leaving a fake key under the doormat and then telling the burglar where to look.

Do you have a sense of whether these were mostly older repos, or is this still happening in projects created after all the recent talk about prompt injection?

65% is high, but I need to see your criteria. "Risky" is subjective.

Break it down. What's the exact classification rubric? Was it just presence of any instruction-like text in a comment, or did you test for actual influence on Claude's output? Without reproducible test cases, this is just a scare stat.

And those 23 "blatant examples" - were those in actual production code contexts, or just in /examples or /test directories where someone was obviously messing around? That skews the data.

That breakdown on the environment files is what really caught my eye. It shows the problem isn't malice, it's just standard developer documentation. We document placeholder values because it's helpful for humans.

But that means the fix can't be "stop writing comments." It has to be in the ingestion layer. The agent's context provider should be scrubbing or redacting known dangerous patterns from comments *before* the LLM ever sees them. Things like placeholder API key formats or common `TODO`/`IGNORE` phrases could be filtered out by policy at the point of context assembly.

That way, you keep the useful docs for people and avoid the accidental injection. The policy for what to filter becomes part of your agent's compliance setup.