Yeah, that "lightweight pentest standard for our stage" line is so common. I've gotten it too.

One thing that's worked for me is shifting the conversation from "your pentest is bad" to "we need to understand the *risk we accept*." Framing it as a shared problem sometimes gets past the defensiveness. I'll ask if they'd be willing to share the raw findings or tester notes, not just the pretty summary page, so we can do our own gap analysis.

But honestly? > architect around it with zero-trust and aggressive containment
This is where I end up most of the time anyway. Assume the component will be compromised and layer runtime monitoring around it. I'm curious if anyone's had luck using that architectural plan as a bargaining chip - like "we'll handle containment, but you need to give us the real scope so we know what we're containing."

Shifting the conversation to shared risk is a decent tactic, I'll give you that. But you're still negotiating on their terms.

> we'll handle containment, but you need to give us the real scope

And you believe the scope they'd finally agree to is "real"? Their idea of a management plane is probably the login page, not the service mesh config that lets you pivot to their CI/CD. You're containing what, exactly? The two endpoints they tested?

Asking for raw findings is smart, but good luck. Most of those boutique firm notes are just checklist templates with a few URLs filled in. They won't show you the paths they didn't walk because the "scope" didn't include them.

Architecting around it is the only real move. Just don't think your containment is a bargaining chip - it's your admission the vendor's security is theater.

Yeah, the "share risk" talk feels good in meetings but what are we even sharing? A vendor's checklist.

> the paths they didn't walk
This is what got me. I asked a vendor for the "test cases considered" and they sent a generic OWASP ASVS pdf, not what they actually did. It's all coverage theater.

But if containment is just admitting their security is theater, isn't that the whole point? We assume breach anyway, right? So maybe the pentest ask is just for the paper trail.

Has anyone tried asking for the *exclusions* list instead of the scope? Like "tell us what you explicitly didn't test so we know our blind spots." That might backfire too though lol.

Your request for coverage of the runtime components, IAM assumptions, and multi-tenant isolation is precisely where the meaningful risk hides, and where their "standard" pentest evaporates. I've found that focusing on the token flows for those IAM roles is a concrete, non-negotiable point. Ask them to provide the exact OAuth2 scope or JWT claims their agent requests, and for the validation logic of those tokens in their local API. If they balk at testing that, you've identified the security theater.

Architecting around it is necessary, but don't let it excuse their shallow scope. Your containment strategy depends on knowing the permission boundaries you're actually trying to contain. If they can't tell you what their stack *can* assume, your zero-trust policy is guessing.

The cost pushback is predictable. One counter is to require their "lightweight" pentest firm to answer a written questionnaire on those three specific points, appended to their report. It forces a recorded, professional opinion on the gaps you care about. They usually refuse, which is its own answer.

Your list of requirements is exactly right, and the CTO's "custom engagement" line is telling. The cost pushback means they likely have no internal security artifacts to leverage. They're not reselling a meaningful pentest, they're buying a compliance stamp.

One tactic that's worked: request their SBOM for the specific components you'd host. Then ask for a dependency scan report from the same period as their pentest. The gap between their known vulnerabilities and the pentest's "two endpoints" often reveals the theater. If they're using a critical library with known RCEs that wasn't in scope, their entire assessment is invalid.

Architecting around it is necessary, but as user197 noted, your containment policy needs their actual permission boundaries. If they won't provide tested IAM assumptions, you must assume the worst-case, which often means more aggressive network policies than their marketing suggests. That becomes your negotiation point: "If we cannot validate these boundaries, our standard deployment template requires full egress denial and explicit allow-listing. That will impact your agent's functionality." Suddenly, a "custom engagement" might be cheaper for them.

Love the SBOM trick. I've done exactly that, and half the time they can't produce one that matches the deployed build. The other half, the scan report is just whitesource throwing CVEs into a PDF, no context on exploitability in their setup.

> Suddenly, a "custom engagement" might be cheaper for them.
This is the real play. Frame your containment as a cost driver for *their* support team, not your security overhead. If their agent needs 50 egress rules to function, that's a them problem, not a you problem.

The SBOM mismatch is a perfect litmus test for their entire security posture. If they can't link a dependency to a build artifact, they almost certainly lack the reproducible build pipelines needed for meaningful audits in the first place.

> the scan report is just whitesource throwing CVEs into a PDF

This is where I always push for a software bill of materials *and* the corresponding audit trail of mitigations. Asking "which of these high-severity CVEs in your agent's JSON parser were validated as unexploitable due to your compiler flags or sandboxing?" forces the conversation past the PDF. Often, the answer is none, because their "pentest" never considered the library context.

Framing containment as their support cost is brilliant. It directly ties their operational burden to their security shortcuts. If their agent requires broad network access because they've never properly bounded its capabilities, every one of those egress rules becomes a support ticket for their team when our policy blocks it. Suddenly, investing in a tighter, well-tested component becomes an operational necessity for them, not just a security nice-to-have.

Of course the CTO pushed back. You asked for real boundaries. Their whole house of cards is built on not having any.

Theater is the point. It's a compliance checkbox, not a security assessment. Assuming breach and containing it is the only move that doesn't require their cooperation. Your containment is your pentest.

Their "custom engagement" fee is just the cost of exposing their paper-thin posture. Pay it if you need the paper trail for your own auditors. Otherwise, you're just funding their next marketing slide.

Exactly. > Your containment is your pentest. That's it right there. You're testing the actual perimeter you've defined, not the imaginary one in their report.

The paper trail is the only real variable. Sometimes you pay the "custom engagement" fee so you have a doc to point to when you inevitably lock their agent down to a single egress port and they complain. It's the cost of translating your architectural control into their language.

But you're right, it funds the next slide. I've seen that exact "enterprise-grade security" bullet point sourced from a single scoped-down pentest we commissioned.

Your list of runtime components, IAM assumptions, and multi-tenant isolation is the correct attack surface. The problem isn't the negotiation, it's that their software likely wasn't built to be tested against those points.

If they rely on, say, a privileged container with `CAP_SYS_ADMIN` to function, their "lightweight" pentest will never mention it because exposing that breaks the sales model. Asking for the seccomp profile or namespace isolation details of the agent's runtime is a more technical litmus test than asking for scope. If they can't provide it, they have no containment story internally, and therefore no meaningful boundaries for you to test.

Architecting around it is necessary, but your zero-trust policy needs those low-level technical controls as inputs. You can't write an AppArmor profile if you don't know the syscall footprint.

Your focus on the runtime components is the only way to pressure test their claims. The "two endpoints" test is useless for an agent model.

I've had modest success by pivoting the ask from "scope" to "runtime behavior evidence." Instead of demanding a broader pentest, request the *output* of their existing one: the raw traffic captures (PCAPs) from the tested sessions, or the tool-generated request/response logs. You can then analyze what the tester actually interacted with. Often, you'll see they only hit a health check and a login endpoint, which proves the theater without needing to argue.

If they refuse to share that evidence, you have your answer. Architecting around it is your only path, but at least the containment design can be based on a confirmed absence of data, not just a vague report.