Just finished the first draft of a threat model for a research agent that autonomously crawls the web. This one feels particularly gnarly—so many ways for it to go sideways. 😅
I'm sharing it here because I think it's a common pattern we're all thinking about. The model focuses heavily on the agent's actions as the primary threat source. Key assumptions include: the agent has read/write access to a local knowledge base, and its instructions can be modified via a web UI. Biggest failure modes I identified were credential harvesting via deceptive sites and accidental DoS against small websites. Would love your eyes on the STRIDE breakdown—am I being paranoid, or not paranoid enough?
- L
Stay safe, stay skeptical.
Focusing on the agent's actions as the primary threat source is the right call, absolutely. That's where the real unpredictability lives.
I'd push you on one of your assumptions though: "instructions can be modified via a web UI." That's a massive, juicy attack surface right there. Your model needs to treat that UI as a core part of the trust boundary. If someone can alter instructions, they own the agent. Consider STRIDE against the instruction pipeline itself - tampering and repudiation are big ones.
On paranoia level? For accidental DoS, you're probably not paranoid enough. Think about the cascading effect if it's recursively following links from a single page. Even with rate limits, the volume from one instruction can be wild.
Agree completely on the threat source focus. The STRIDE breakdown for the agent's actions is crucial, but I'd extend it to the data flows from the knowledge base, too. If the agent can write to it, poisoned or misleading web data gets a permanent foothold.
For credential harvesting, have you considered the parsing step? An agent scraping a page might extract what looks like a login form or an API key pattern. Even if it doesn't "submit" anything, just storing that extracted string could be a violation. Your model should cover data processing, not just network calls.
On paranoia: for accidental DoS, user484 is right. Implement a per-domain crawl delay, but also a total request budget per instruction. And make sure those limits are in the threat model as a security control - otherwise they're just a reliability feature.