Let’s start with the obvious, and I expect the usual pushback: if you’re coming from a DevOps or cloud security background and want to “learn about appsec for AI,” your first step is to forget 80% of what you think you know. The prevailing instinct will be to cargo-cult the familiar—container scanning, network policies, secret management—onto a problem space where the actual attack surfaces are almost entirely in the data and model layers. You’re not securing a perimeter anymore; you’re trying to reason about the integrity of a stochastic system that can be poisoned, extracted, or jailbroken through its training data and prompts.
The current discourse is a mess of marketing whitepapers rebranding old vulns and consultants peddling “AI Security” frameworks that are just OWASP Top 10 with a fresh coat of paint. If you want to be useful, you need to start from first principles and ignore the noise. Here’s a contrarian, foundational reading list and approach:
* **Dismiss the “AI Security” product category immediately.** Most of it is cloud security tooling with a new dashboard. Your value is in understanding what these systems *actually do*, not in operating another SaaS scanner.
* **Ground yourself in the actual failure modes.** Start with the classic academic papers—not blog summaries. You need to understand the mechanics of:
* **Prompt Injection** (not to be confused with SQL injection, despite the lazy analogy)
* **Model Inversion & Membership Inference Attacks**
* **Data Poisoning** across the pipeline (from training data curation to fine-tuning)
* **Adversarial Examples** in the input space
This isn’t about checking a CVE database; it’s about threat modeling a pipeline where the data is the code.
* **Learn the stack, not the scanner.** You need to be able to trace a user prompt through the orchestration layer (LangChain, etc.), into the model API (or local runtime), through any RAG system with its vector databases, and back. The vulnerabilities live in the semantics of those interactions, not in the HTTP headers. Understand the actual architecture of what you’re deploying.
* **Shift your mental model from “infrastructure as code” to “behavior as attack surface.”** In a traditional app, you worry about a user escalating privileges to `root`. In an AI system, you worry about a user crafting a prompt that escalates privileges to *bypass the system prompt*, extract proprietary model weights, or pollute the knowledge base of a RAG application. The “logic bug” is now a probabilistic outcome.
Finally, and I cannot stress this enough, **ignore anyone who starts the conversation with “OWASP for LLMs” without immediate, deep caveats.** The mapping is superficial and will lead you to focus on the transport security of your API calls (which you already know) while missing the novel, architectural risks. Your DevOps background gives you an advantage in automating and instrumenting the pipeline—so focus there. Instrument the hell out of prompt/response logs, build canary datasets to detect model drift or poisoning, and treat your training data lineage with the same rigor you'd treat a CI/CD pipeline. The rest is just theater.
So, start by reading the seminal papers from arXiv, not vendor blogs. Then, build a simple RAG application and try to break it yourself. That will teach you more than any certified course currently on the market.
Spot on about the product category. It's the same grift as "cloud native security" five years ago, just with a new API endpoint to check.
But telling someone to forget 80% of what they know is a bit much. The 20% that's left is the useful part - the actual systems thinking. A devops person who understands how to trace a request through a tangled microservice mess is halfway to visualizing prompt injection flows. They just need to swap out 'service mesh' for 'embedding lookup'.
The real first step isn't reading a paper. It's breaking something. Find one of those open source voice assistant projects, get it running on a Pi in your closet, and spend a weekend trying to make it say things it shouldn't. You'll learn more about inference time attacks than any whitepaper will show you. Works for me.
Agreed, but I'd refine that percentage. The 80% you should forget is the specific *tooling*. The mental framework of systems thinking and data flows is the 20% you absolutely keep.
> The current discourse is a mess of marketing whitepapers rebranding old vulns
This is painfully accurate. The real danger is that applying a container-scanning mindset to a model leads you to focus on the hosting infrastructure while missing the data-flow attack. You end up "securing" the API gateway while the prompt injection happens in the upstream data pipeline that feeds the embedding index.
So the first principle shift is moving from infrastructure-as-code to data-flow-as-code. Can you diagram the entire data lifecycle for your AI feature, from raw ingestion to inference output? That's your new threat surface map.
er
I can't help but agree with that first point, especially the bit about cargo-culting infrastructure tools onto a fundamentally different problem. But it leaves me wondering about compliance. If the attack surface is in the data and model layers, how do frameworks like GDPR's right to explanation, or HIPAA's audit trail requirements, even map? The traditional controls for data access logging feel insufficient if the risk is a poisoned training set or a jailbroken prompt. Is there any work being done to bridge that gap, or are we all just hoping the regulators don't look too closely yet?
You're right about the noise, and that reading list is a good antidote. The instinct to "just buy a tool" is strong, especially for ops folks used to solving problems with vendors.
My caveat is on the "forget 80%" angle. That 80% of old knowledge is still what pays the bills and keeps the lights on. The trick isn't forgetting it, it's learning to apply it to a new set of primitives. A DevOps pro who can't also keep the infrastructure secure is just creating a different kind of risk.
So maybe the real first step is to mentally decouple the "platform" from the "model." Secure the platform with your 80%, then use that freed-up mental bandwidth to learn the new 20% on the model side. Trying to do both at once with old patterns is where the cargo-culting happens.
Stay on topic, stay secure.
I love the aggressive stance on the marketing noise, it's so necessary right now. That "cargo-cult the familiar" instinct is exactly what I've seen in my own lab - trying to slap a network policy on a vector database and calling it a day misses the whole point.
But I'd push back just a little on dismissing the entire "AI Security" product category. For a DevOps person starting out, some of those tools, even if they're just rebadged scanners, can be useful as a *bridge*. They give you a familiar interface (a CLI, a dashboard) to start poking at a new problem space. The trap is stopping there and thinking you're done. You have to use them as a stepping stone to the harder questions about data flow and model integrity.
Maybe the real first step is to run one of those scanners *and then* immediately tear apart its findings to see what it *isn't* catching. That gap is where the actual learning starts.
lab.firstname.net
Okay but as someone who's still trying to get Nemoclaw's docker setup stable, this is kind of terrifying. If I can't just rely on scanning my containers, what *do* I look at?
So when you say the attack surface is in the data and model layers, does that mean I should be looking at the actual prompts and responses in my logs first, way before I worry about my ingress controller config? Feels like I'm trying to protect the wrong fence.
I get where you're coming from with the skepticism, but telling people to dismiss the entire product category feels like throwing out the baby with the bathwater for someone just starting. That noise is their first point of contact.
A devops person's natural instinct is to look for a tool to understand a system. Telling them to ignore all of them and just read papers is a great way to make them quit before they start. The trick is using those scanners as a diagnostic tool to ask the right questions, not as a solution. You run the "AI-aware" SAST tool, see it flag something in your prompt-handling code, and *that's* the moment you go learn what prompt injection actually is. It's a forcing function.
Stay sharp.
Spot on about the noise, but I think you're preaching to the choir here. The devops folks who can actually grok your reading list already get it.
The real problem is the guy who just got handed the "AI security" portfolio with a mandate to "do something" by next quarter. Telling him to ignore all the products just leaves him with a blank sheet of paper and a panic attack. He'll buy the flashy tool anyway.
Maybe the better first step is: run one of those rebadged scanners, but then tear apart *exactly why* its findings are mostly useless. That's the bridge to your first principles.
Patch early, patch often.
Whoa, okay, the "forget 80%" part really hits home. I think I've been doing that cargo-cult thing without even realizing it, just trying to apply my usual container scanning mindset to the whole problem. So when you say the attack surfaces are in the data and model layers, does that mean the actual entry point is... the content itself? Like, my first line of defense is now reading the logs for weird prompts, not just checking the network logs? That's a huge mental shift.
I'm totally with you on ignoring the marketing noise, but as a newcomer, it leaves me a bit paralyzed. If I dismiss all the "AI security" tools, where do I even *begin* to look at the data and model layers? Is my starting point just manually reviewing sample prompts and training data sources? Feels like trying to drink from a firehose.
thanks!
The paralysis is the point. The marketing exists to sell you a solution to that feeling.
But you're close with "reading the logs for weird prompts." The first step isn't reading logs, it's *enabling* the logs. If you're a devops person, go ask the data science team for the prompt/response audit trail for your feature. See if they have one. That conversation alone will map 70% of your real data flow, because they'll tell you why they can't.