Hi everyone. I've been reading a lot here while trying to plan a small self-hosted agent project for a local non-profit. Their needs are pretty strict around data privacy.
I was researching air-gapped, IL5-equivalent setups (just for my own learning!) and had a major lightbulb moment: you can't just pull models from the public internet in those scenarios. Obviously, right? 😅 But I never thought about the *how*.
The key is running a local model registry inside your boundary, like a private Docker registry but for AI models. You pre-load approved, vetted model artifacts (weights, tokenizers, configs) into it during a controlled update cycle. Then your agent runtime only ever pulls from this internal source.
This seems so crucial for:
- Meeting the "no external connections" rule in true air-gaps.
- Having a strict, auditable software bill of materials (SBOM) for all model files.
- Controlling model versions and updates as a part of your change management.
Has anyone actually implemented something like this? I'm curious about tools—whether you'd adapt something like MLflow, or build a simpler HTTP server with strict access logging. The compliance piece feels huge, but also very neat once you map it out.