Skip to content

Forum

AI Assistant
Notifications
Clear all

Audit logs are ballooning to 100GB/day, can't find anything. Help?

17 Posts
17 Users
0 Reactions
2 Views
(@selftaught_sec)
Active Member
Joined: 1 week ago
Posts: 11
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

I completely agree about separating the loggers at the framework level. A dedicated `log_security_event` function that mandates extra fields is the right direction. But that brings up a question about scope creep - what qualifies as a "security event" that gets forced into that stream?

I've seen teams start with database calls, then add user authentication, then file system access, and before you know it, the "secure" stream is just a renamed version of the old bloated log because everything feels important. The framework needs a brutally strict definition, maybe tied directly to a data classification label or a specific resource namespace.

And on dropping args for `get_weather`, you're right about complexity. But doesn't that just move the problem to defining the retention window? If a buggy plugin starts hammering the API, you'd want those raw arguments to debug the surge, but they'd be gone if your window is too short. How do you decide what's short enough for ops but long enough for incident response?



   
ReplyQuote
(@newbie_shield)
Eminent Member
Joined: 1 week ago
Posts: 21
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Scope creep is exactly why our team gave up on trying to define "security events." It's always "just this one more thing." 😅 We ended up using the data classification label idea you mentioned. If a function touches data labeled "confidential" or "restricted," it goes to the secure stream. Everything else goes to the ops drain. It's not perfect, but it's a hard line.

For the retention window, we got burned by a short one during a DDoS. Our ops logs for the affected service were gone. Now we keep a tiny sample (like 0.1%) of all low-sev calls forever, just the metadata. The raw args still vanish after 24h, but we at least have a record that *something* spiked. It's a cheap compromise.



   
ReplyQuote
Page 2 / 2