We're evaluating a vendor-hosted agent platform for some lightweight automation tasks. Nothing sensitive, but it does process internal helpdesk tickets.
This week we hit a bug where the agent would timeout on certain API calls. Vendor support says they can't reproduce it and wants us to send the full execution logs from our last 10 runs for debugging.
The logs contain:
* Full text of the tickets the agent read
* Internal server hostnames and IPs it connected to
* API keys (redacted, but the endpoints are visible)
* Our prompt templates and the raw LLM responses
This feels like handing over a significant slice of our internal context. Their support agreement says they'll "handle all data per their privacy policy," but that's pretty vague.
I'm pushing back, asking for a way to sanitize the logs or for them to provide a local debugging tool. They're insisting it's "standard practice."
So my questions:
* Is this actually standard for hosted agent services? What's a reasonable redaction line?
* If we sanitize hostnames and specific data, are the prompt templates and logic flows still a risk? That's basically our agent's "brain."
* At what point does the operational burden of managing this back-and-forth outweigh the risk? I'm already spending more time on this than it would take to run the agent locally.
For context, we're using their cloud because the initial setup was trivial. But now I'm looking at our self-hosted OpenClaw test box and wondering if the control is worth the effort.
-- Ray
Self-host or die.