I've been looking at both OpenHands and SuperAGI for some local AI agent workflows, but I'm stuck on one particular security concern before I even think about deployment. The documentation for both talks about "local" or "self-hosted" operation, but I've seen enough tools that still make unexpected calls home even when you think they're contained.
My main question is: which of these frameworks gives you better, more granular control over outbound network connections? I'm especially concerned about the agent itself trying to reach out to external APIs or services without explicit configuration on my part. I run everything in an isolated VLAN, but I'd prefer the software itself to be designed with a default-deny stance for external calls.
From my reading, OpenHands seems to emphasize its local-first architecture, but I couldn't find a clear list of what domains or IPs it might attempt to contact during normal operation. SuperAGI's setup mentions environment variables for API keys, but does it fail closed if those aren't set, or does it try some fallback?
Has anyone done a packet capture or set up strict egress firewall rules while testing either platform? I'm less interested in feature comparisons right now and more in understanding the actual network behavior. Knowing which one is more transparent and controllable would help me decide where to invest my testing time.
Paul
Better safe than sorry.
That's a really good point about the default-deny stance. I'm also trying to wrap my head around this for my own setup. I haven't done packet captures myself (that's a bit beyond me right now), but I did test OpenHands in a Docker container with no internet access.
In my test, the core agent itself just sat there idle without making calls, which was good. But the moment I tried to use a tool that needed the web, like the research function, it just failed with a generic connection error. It didn't seem to have a built-in allowlist for outbound calls, it just tried and died. So the control feels more about not giving it tools that can call out, which is kinda manual.
For SuperAGI, I saw someone in their Discord mention that if you don't set the OpenAI API key (or whatever LLM you're using), the agent just fails to initialize. So it fails closed on that front, which is good. But I'm not sure about other services it might bundle. Have you found any mention of a network configuration file for either one? I feel like that would be the real test.
That's exactly what I'm worried about. If the framework itself has a fallback or telemetry baked in, network rules become a cat and mouse game.
You mentioned a packet capture would help. I'm new to this, but would running it in a fresh VM with Wireshark actually show you the attempted connections, even if they fail? Or does the software need to actually succeed to log the destination?
I'm leaning toward trying OpenHands based on your test, since a hard fail is what I want. But I'm still nervous there's a hidden "phone home" for error reporting or something.