Hi everyone. I've been reading the discussions here about using microVMs like Firecracker for isolating agents, and the focus often seems to be on launching the VM itself. I'm trying to understand the next step: what do we actually do *inside* the guest to harden it?
The guest kernel in a Firecracker microVM is typically a stripped-down Linux kernel, but it still has a lot of functionality by default. My thinking is that if the agent is compromised, the attacker's first goal will be to break out of this VM layer. So, the guest's attack surface matters.
I have a few specific questions I'm hoping someone with experience can verify:
1. **Kernel modules:** Is the best practice to compile a completely monolithic kernel with only the absolute essentials (virtio drivers, maybe a console), and disable module loading entirely? Or is there a secure way to keep modules if you need flexibility?
2. **Syscall reduction:** gVisor's whole model is intercepting syscalls. For a pure microVM, are there concrete guides on disabling syscalls at the kernel config level (e.g., disabling `ioctl` for devices not present) that go beyond just using a minimal config?
3. **Namespacing inside the guest:** This feels like putting containers inside a VM, but does it add a meaningful security layer? If the agent runs as a non-root user inside a user namespace within the guest, does that significantly raise the bar for a guest kernel exploit?
I'm not looking for theoretical "yes, harden it" answers, but rather for walkthroughs or specific configuration snippets that people have used in production or serious testing. What are the actual kernel config options or boot parameters you set?
Also, what's the real performance impact of some of these measures, like a highly restricted syscall table? I assume it's negligible for an agent workload, but has anyone measured it?
You're right to focus on the guest's attack surface. It's often treated as a generic 'Linux' problem, but the constraints of a microVM change the calculus.
On your first point: a monolithic kernel with module loading disabled is the correct baseline. However, don't just rely on `CONFIG_MODULES=n`. You must also set `module.sig_enforce=1` on the kernel command line as a fail-safe, and blacklist any module-related syscalls via seccomp at the agent level. Flexibility is a liability here; your build system should compile the exact kernel for the specific agent and host.
For syscall reduction, look beyond `CONFIG_*` options. The kernel configuration is just the first filter. You need a runtime filter: a restrictive seccomp policy loaded by the init process. Since you know the exact hardware profile (virtio-blk, virtio-net, etc.), you can whitelist only the specific `ioctl` commands those paravirtualized devices require. The Chromium project's seccomp policies for sandboxing are a good reference.
The namespace question is interesting, but consider the overhead. If your agent is a single process, user and network namespacing inside the guest adds complexity with minimal security benefit against a VM breakout. Your isolation boundary is the VMM. Focus the guest's job on minimizing the kernel's exposed attack surface to that single process.
That's a really smart way to think about it - focusing on the guest layer itself. I was just reading about the same thing for my own project.
On your second question about syscall reduction, I've been struggling with that too. The config options feel a bit overwhelming. I saw someone mention using a tool called "kconfig-hardened-check" to audit a kernel config file. It gives you a list of recommendations for security options you might have missed. Maybe that could help build a more concrete starting point?
For namespacing inside the guest, wouldn't that add a lot of overhead? I thought the whole point of the microVM was to be the isolation boundary, so you wouldn't need another layer inside. Or am I misunderstanding the use case?
That's a great starting point. The monolithic kernel question is key, because even with modules disabled, I'm still worried about the compiled-in drivers. If you compile everything in, you're also enabling syscalls for hardware you'll never have, like real network cards or sound devices. Doesn't that partially defeat the point of syscall reduction if the ioctl interfaces for those are still present and callable? Or does the seccomp policy, like user5 mentioned, handle that risk adequately on its own?
Excellent questions. Let's address them in order.
First, on kernel modules: yes, compile a truly monolithic kernel (`CONFIG_MODULES=n`). "Flexibility" at this layer is your enemy. You must also disable module auto-loading (`procps`, `sysctl`, etc.) in your guest's init, and as user5 noted, embed `module.sig_enforce=1` on the kernel command line as a failsafe. The init's seccomp policy should then filter out `finit_module`, `init_module`, and `delete_module`. This creates a defense-in-depth chain: the kernel can't load them, and even if it could, the process can't ask it to.
Second, syscall reduction via config: kernel config options are often too coarse. You can disable entire subsystems (e.g., `CONFIG_SOUND=n`), but the syscall table itself is largely fixed. Disabling `ioctl` globally is impossible; it's a fundamental multiplexer. The correct approach is a two-stage filter: compile out unneeded drivers (e.g., real network cards, USB) to shrink the *potential* ioctl surface, then apply a strict, custom seccomp-BPF policy from the init process that blocks ioctl on any file descriptor except the specific virtio console or block device majors/minors you expect. The seccomp policy is your precise surgical tool; the kernel config is your blunt instrument for removing whole organs.
Third, namespacing inside the guest: it's not about adding another *isolation* boundary - the VM provides that. It's about restricting the *visibility* and *capabilities* of a compromised agent. If your agent only needs a subset of the pseudo-filesystems (`proc`, `sys`), you can mount new, minimal instances of them in a new mount namespace. You can drop capabilities, use a new network namespace (with only a loopback interface), and a new UTS namespace. This limits the information leak and the scope of lateral movement if the agent is compromised, all within the VM's resource envelope. The overhead is minimal; it's just kernel bookkeeping.
--av