Skip to content

Forum

AI Assistant
Notifications
Clear all

Step-by-step: Auditing the supply chain of an AutoGen plugin from PyPI to runtime

1 Posts
1 Users
0 Reactions
3 Views
(@bob_hardcase)
Eminent Member
Joined: 1 week ago
Posts: 16
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
  [#228]

Hey everyone, been lurking for a bit but wanted to jump in. I've been experimenting with a bunch of AutoGen plugins from PyPI lately, and it got me thinking: how do we *really* know what we're installing? The docs say one thing, but the code could be doing another, right?

I'm trying to build a repeatable audit process for myself, but I feel like I might be missing steps or overcomplicating things. Here's my current checklist when I find a promising plugin, say `autogen-plugin-awesome-tool`:

* First, I check PyPI for the package page and the listed dependencies. But the metadata there is just what the author submitted.
* So I pull the source distribution (.tar.gz) and the wheel, diff them if they exist. Look for hidden scripts or odd binaries.
* Then I scan the `setup.py` or `pyproject.toml` for custom build steps or post-install hooks.
* I review the actual source, especially the `__init__.py` and any module that handles external API calls. Look for obfuscated code or weird `eval()` calls.

But is that enough? Why not just rely on the hash from PyPI? I feel like a determined actor could still slip something past this. For example, what about:
* The plugin's own dependencies? Do I need to audit *their* supply chain too? That's a rabbit hole.
* Dynamic code loading from the web? I've seen `requests.get()` + `exec()` patterns in other contexts.

My main skills are in Python scripting and API integration, so maybe I'm overestimating how deep I need to go for a typical agent framework plugin. What steps do you all take before you `pip install` something that's going to have access to your LLM calls and potentially external tools? Are there any automated scanners you'd recommend, or is manual review still the only real way?



   
Quote