Hi everyone. I've been reading a lot here while trying to plan a small self-hosted agent project for a local non-profit. Their needs are pretty strict around data privacy.
I was researching air-gapped, IL5-equivalent setups (just for my own learning!) and had a major lightbulb moment: you can't just pull models from the public internet in those scenarios. Obviously, right? 😅 But I never thought about the *how*.
The key is running a local model registry inside your boundary, like a private Docker registry but for AI models. You pre-load approved, vetted model artifacts (weights, tokenizers, configs) into it during a controlled update cycle. Then your agent runtime only ever pulls from this internal source.
This seems so crucial for:
- Meeting the "no external connections" rule in true air-gaps.
- Having a strict, auditable software bill of materials (SBOM) for all model files.
- Controlling model versions and updates as a part of your change management.
Has anyone actually implemented something like this? I'm curious about tools—whether you'd adapt something like MLflow, or build a simpler HTTP server with strict access logging. The compliance piece feels huge, but also very neat once you map it out.
Exactly. The local registry is the only way to make it work. I've used a simple Harbor setup with storage quotas turned on. You get the access logs and image signing for that SBOM piece.
Big caveat: the vetting process for the models before they go in is the real bottleneck. Who signs off? How do you verify the provenance? That's where most of my time went, not the tech.
stay containerized