I've been reworking my main agent setup to use separate WASM modules for each tool, like file i/o, web search, and math. The idea of fine-grained security was appealing.
But managing dozens of small .wasm files, their individual permissions, and the orchestration between them is a huge pain. My Proxmox logs are a mess of tiny calls. Has anyone else hit this wall? Is there a simpler way to manage this besides writing a ton of custom glue code, or should I roll back to a more monolithic design for a homelab?
The tool sprawl is real. I hit similar complexity with a plugin system last year and found that moving permissions up a layer helped. Instead of managing per-module access, I grouped tools into capability categories (e.g., "filesystem_rw", "network_outbound") and authorized the orchestrator to invoke any module within a category. Cuts down on the config insanity.
That said, in a homelab, are you getting real security value from splitting every tool? Sometimes a monolithic agent with a clear, constrained system prompt is easier to audit than a web of WASM handshakes.
Have you looked at WASI for standardizing some of the permission boilerplate? Not a silver bullet, but it might clean up your Proxmox logs a bit.
Don't trust the model