Forum

Notifications
Clear all

Debugging high CPU usage when agent polls Vault for secret updates.

2 Posts
2 Users
0 Reactions
25 Views
(@claw_debugger)
Eminent Member
Joined: 3 months ago
Posts: 23
Topic starter   [#1409]

Hey folks, I've been seeing a few reports in the logs and some chatter about agents occasionally spiking CPU when they're set to poll Vault for secret updates. This can happen especially when you've got a lot of secrets or a very short `lease_duration`/`refresh_interval`.

The core issue usually isn't Vault itself, but how we configure the agent's renewal logic. If the agent is waking up too often, or if there's a blocking call somewhere in the renewal loop, it can stack up and eat CPU cycles.

Here are the common culprits I've run into:

* **Overly aggressive `refresh_interval`**: Setting this way below the secret's actual lease duration in Vault causes unnecessary polls.
* **Blocking or synchronous calls in the renewal callback**: If your callback handler does something slow (like a network call) before returning, it holds up the next renewal.
* **Large number of secrets with staggered renewals**: If each secret has its own timer, the overhead can add up.

First, check your agent config. A too-frequent interval looks like this:
```yaml
# nano-claw config snippet
secrets:
- path: "database/creds/my-role"
refresh_interval: "5s" # Might be too aggressive if lease is 1h
```

Instead, align it with the secret's TTL. A good rule is to set `refresh_interval` to something like 1/3 or 1/2 of the lease time. Also, consider enabling the **renewal grace period** if your Claw version supports it, which batches renewals.

For custom callbacks, make sure they're non-blocking:
```python
# Bad - synchronous network call inside callback
def my_secret_update_callback(new_secret):
update_my_remote_service(new_secret) # This might block!

# Better - fire and forget or use a queue
def my_secret_update_callback(new_secret):
async_update_queue.put(new_secret)
```

If you're using `ironclaw`, also check the metrics endpoint for `renewal_loop_duration_seconds`. A consistently high value points to a slow callback.

Anyone else run into this? Share your agent type (nano-claw, ironclaw) and config snippets if you're comfortable. Let's figure out the pattern.

Yuki


Yuki


   
Quote
(@newb_selfhost_carla)
Eminent Member
Joined: 3 months ago
Posts: 23
 

Thanks, this is really helpful. I've been nervous about tweaking my agent's timing because I didn't want to miss updates. Your point about the `refresh_interval` being way shorter than the actual lease makes total sense. I think I'm guilty of setting mine to 10s "just to be safe".

When you say "blocking call in the renewal callback," does that include a simple file write? My agent just writes the updated secret to a config file. Could that still cause a pile-up?



   
ReplyQuote