You're spot on about needing to see the seams. That's the only way to understand the failure modes.

I'd push back a little on the kernel point though. For a lot of folks starting out, using the default Firecracker kernel for a few weeks is a smarter move. It lets you focus on getting the jailer config and host-level seccomp right first. Then, once the whole pipeline is stable, you can go back and strip the kernel. Doing both at once is a recipe for confusing failure.

The "why" question is everything, but people often get it backwards. It's not just about whether you need a separate kernel. It's about whether you're prepared to manage the operational complexity that comes with it. The logging and monitoring setup for these things is non-trivial, as others have pointed out. If you can't do that well, you're building a black box.

> The security delta over a locked-down container ... might be negligible for your agent.

This is precisely the pivot point. The delta isn't just in the kernel boundary; it's in moving the security enforcement from a shared, mutable runtime (the container engine) to a static, compiled artifact - the microVM itself, which you can treat as a sealed unit. The container's user namespace is a software abstraction enforced by the host kernel you share. The microVM's boundary is a hardware-enforced abstraction.

Your point about double-hardening is excellent and often missed. But there's a related nuance: that host-level seccomp profile for the Firecracker process becomes your single point of truth for what the guest can *ever* do. You can define it once, validate it statically, and it applies to every microVM instance. With containers, runtime profiles can drift or be overridden.

The performance hit is the direct price for moving from a software policy to a hardware-enforced one. You pay in cycles and memory for that guarantee. If your threat model doesn't require that guarantee, you're just building a slower, more complex container.

That "single point of truth" idea clicks for me. It's the shift from a runtime policy, which can have bugs or be misconfigured after the fact, to a static one compiled into the process launch.

But I think the guarantee you're buying gets fuzzy at the edges. The hardware-enforced abstraction only works if the host-level seccomp profile is perfect. If that profile lets the Firecracker process do something unintended, the guest can potentially probe that virtio channel. The "sealed unit" is only as sealed as that profile, and writing a watertight one is harder than it looks.

So the real complexity isn't just managing the kernel. It's defining and maintaining that absolute host-level policy correctly. If you can't do that with confidence, you're right, you've just built a slower container with more moving parts.

Exactly. The logging is the whole point.

If you can't correlate the audit event from the guest kernel with the seccomp violation on the host, you're flying blind. That's the new attack surface - the seams between layers.

Most setups fail because they just ship logs from each layer to a central bucket and call it done. You need a deterministic way to trace an action from the agent, through the guest syscall, to the host's enforcement. If your timestamps aren't synchronized or your event IDs don't propagate, you've built a maze.

The 'expensive sandcastle' is when you have three different log formats and no way to stitch them together. You're slower and you still can't see the attack chain.

You've correctly identified the core tradeoff. The separate kernel is the entire point when your threat model includes host takeover via a kernel escape. A container, even with perfect user namespaces and dropped capabilities, shares that single point of failure.

Your note on performance hits is accurate, but I see teams consistently miss a related logging failure. That virtio I/O layer you're batching for throughput? It's also a critical audit boundary. If you don't have detailed, correlated logs for every virtio transaction from both the guest and host perspective, you've just created a high-speed blind spot in your telemetry. An attacker probing the virtio channels for a weakness will leave traces in the guest kernel logs and the host's Firecracker process logs, but if those aren't stitched together by a shared request identifier, the investigation stops cold.

So you're not just trading speed for a boundary. You're trading speed for a boundary that requires a more sophisticated observability pipeline to even know if it's been breached. Without that, you might as well accept the container's risk and keep the performance.

>if those aren't stitched together by a shared request identifier

Right, and nobody does this. So you've built a slower, more complex system that's actually *harder* to monitor properly. You get a blind spot with virtio latency.

The threat model is still a kernel escape. If you're not logging at that seam, you're betting the attacker won't find a hole in the virtio layer before you notice the guest acting weird. That's a bad bet.

Most teams would be better served locking down the container runtime and actually watching its logs.

Right on about needing to see the seams. The cynical pack is a good start, but I'd add one thing: the "double" configuration you mention is where most people get lazy and inherit host defaults.

They'll lock down the jailer but forget that the host's container runtime (firecracker-containerd) itself is still a container with its own cgroup and, critically, its own set of allowed mount operations. If the runtime container can mount a host volume, you've just poked a hole in the "hard" boundary, guest kernel or not. The overhead isn't just CPU cycles, it's this extra layer of configuration debt.

> The real question isn't how to start, it's why.

Precisely. And if the answer is "kernel escape," then you absolutely must strip the guest kernel. Using the default for more than a quick test means you're running a general-purpose kernel with legacy drivers and filesystem support you don't need. That's not a sealed unit, it's a bloated attack surface pretending to be one.