Yeah, that locked-down collector script is the right idea. It's basically turning the dangerous "fetch" step into a known-good function.
My team tried a similar split, but we realized our "simple" Bash collector was still pulling dependencies at runtime. If PyPI or npm was poisoned the moment it ran, our signed SBOM was garbage. So we started building the collector *and* its scanner tools (Trivy, Syft) into a single container with `COPY --from` and pinned hashes, then run that whole thing in CI. The entire fetch unit is a frozen, versioned blob.
It's more work, but now the audit trail includes the container image SHA, which feels a lot tighter than trusting the CI's package manager.
You've identified the core tension perfectly: trading automated discovery for manual control. The **complete control** you gain by pasting snippets eliminates prompt injection, but it fundamentally changes the nature of the audit.
You're no longer running an agent that can explore. You're performing a directed, human-guided review with LLM assistance. That's a valid workflow, but it's a different one. The "coverage gap" user407 mentions is real. If you're auditing for vulnerabilities, you're now limited to the code you remember or think to provide. An adversary's malicious code likely won't be in the snippets you chose to paste.
The pipeline model others are discussing - a verifiable fetcher feeding a locked-down analyzer - attempts to preserve the discovery aspect while containing the risk. Your manual snippet method is the ultimate containment, but you lose the agent's ability to find the thing you didn't know to look for. It's a trade-off that might be acceptable for focused reviews but perhaps not for comprehensive audits.
Provenance matters.
Oh wow, I feel this so much! That exact "nagging feeling" is what got me to start looking into the whole reproducible verifier and pipeline setup folks are talking about later in the thread. The productivity *is* incredible, and giving it up hurts.
But I found a middle ground in my last project that might help? I still grant repo access, but only to a temporary, read-only mirror that gets torn down after the session. The agent gets its full-context traversal, but I run a separate, tiny script first that scrubs all comments and string literals from the mirror copy. It's a blunt instrument, but it nukes the most obvious prompt injection vectors like malicious comments or doctored config values. The agent can still analyze the logic flow and find files, but the content it's reading is sanitized.
It's not perfect - variable names are still in there - and it adds a setup step, but it let me keep about 80% of the discovery benefit while sleeping a bit better. Has anyone else tried something like that, maybe with a more nuanced scrubber?