Perplexity scoring on raw user input? Sounds like another academic solution looking for a real problem. We tried a prototype. The false positive rate was a nightmare—every industry-specific term, every typo, every non-native speaker got flagged.
What's your actual risk model? Are you trying to catch zero-day injection patterns? Because for compliance, I need a deterministic audit trail, not a probability score. How are you handling the explainability requirement when your heuristic black box blocks a legitimate customer transaction? The regulatory bodies won't care about your clever math.
Compliance is security.
The false positive problem is real, but it's often a result of applying a general language model to a domain-specific input stream without adaptation. Your point about industry terms and non-native patterns is correct. The scoring needs to be contextualized, which means training or fine-tuning on your own legitimate transaction logs to establish a baseline for "normal" utterances in your system.
Your question about a deterministic audit trail is the critical one. A pure probability score is insufficient for compliance. The implementation must produce a discrete, explainable reason for a block. One approach is to couple perplexity with a rule-based classifier: only flag inputs that are both high-perplexity *and* match a structural anomaly pattern, like a sudden shift from natural language to a formatted payload. The perplexity score then becomes supporting evidence for a deterministic rule violation, not the primary trigger.
Without that hybrid model, you're right, regulators will see it as a black box. The math has to serve the audit log, not replace it.
> every industry-specific term, every typo, every non-native speaker got flagged.
This is what worries me. I'm still learning about this stuff, but wouldn't you need a model trained only on your normal traffic to even have a chance? Using a general model seems like it would always have this problem.
Also, your point about the audit trail is huge. If the system says "blocked because 83.4% perplexity score," how do you explain that to a person? It just sounds like "computer says no."