I've been running a small project that uses an agent framework—nothing fancy, just some Python scripts that orchestrate a few LLM calls for a home automation dashboard. Like everyone else, I've been paranoid about the dependency tree, especially with all these new packages popping up in the LLM ecosystem that seem to get updated hourly. My requirements were straightforward: I needed Software Composition Analysis that could actually understand transitive dependencies in a Python project, and I wanted it to run automatically on every commit and pull request.
For the longest time, I used GitHub Actions with a combination of `pip-audit` and Dependabot. It worked, but it felt... clunky. The alerts were often noisy, and pinning strategies were a manual headache. So last month, I decided to switch the entire project over to GitLab CI, lured by the built-in dependency scanning feature they tout. The setup was indeed cleaner in some ways—a single `.gitlab-ci.yml` file with a `dependencies` job using their `gemnasium` analyzer.
But now I'm having second thoughts. The GitLab scanner seems to have a much narrower CVE database for Python packages compared to what I was pulling from the OSV database with `pip-audit`. I'm missing vulnerabilities on indirect dependencies that GitHub Actions/Dependabot would flag. More critically, it seems completely oblivious to the concept of "malicious package" detection, which is a huge part of the threat model when you're pulling in niche, high-velocity packages from PyPI for an agent system.
Has anyone else made a similar move? I'm trying to understand if I've misconfigured something, or if this is a known limitation. I'm also curious about how people are handling the *pinning* aspect in CI. GitLab's scanner gives you a report, but it doesn't automatically generate patches or PRs for updates like Dependabot does. I'm now writing my own scripts to parse the JSON output and create merge requests, which feels like a step backwards. The promise of an integrated pipeline was appealing, but I'm wondering if I've just traded one set of problems for a more opaque, less capable set.
Yeah, that's interesting to hear. I'm still on GitHub Actions for my stuff, so I've been wondering about making a similar switch. When you say a narrower CVE database, do you mean it's missing stuff for newer LLM-related packages, or is it a general lag across the board? That sounds like a big deal.
The database scope is a known limitation of some integrated scanners. They often prioritize coverage for a core set of languages and ecosystems, and newer, fast-moving spaces like LLM tooling can fall behind.
You might find the results improve if you explicitly generate a detailed SBOM (using something like `cyclonedx-bom` for Python) and feed that into the GitLab scan, rather than relying solely on its runtime analysis. It gives the analyzer a complete bill of materials to check against its database.
But if the underlying CVE feed itself is lacking, that's a harder problem. You're likely seeing a general lag.
SLSA >= 2 or go home