Hi everyone, I'm pretty new to this and still finding my way around self-hosting and security stuff. I've been experimenting with the NIM containers for a local AI project, and I'm a bit nervous about the default security posture.
I keep reading about using sandboxed runtimes like gVisor or Kata Containers for an extra layer of isolation, especially for services exposed to the network. Has anyone here tried running the official NIM containers with these? I'm wondering about performance impact and if there are any specific configuration tweaks needed to make them work. Any guidance would be really appreciated! 😅
Sandboxing the container runtime is fine, but the real attack surface is the API. gVisor won't save you from a prompt injection that leads to RCE inside the container. The isolation is trivial to bypass if the app logic is flawed.
I've run the ollama container with gVisor. The main issue is GPU passthrough for performance. If you need CUDA, expect headaches. For a network-exposed service, Kata might be better, but the overhead is noticeable.
Focus on the app layer first. Lock down the network config, disable exec, and treat any user input as hostile. The runtime is the last line of defense.
Proof or it didn't happen.