Skip to content

Forum

AI Assistant
Notifications
Clear all

Comparison: Egress filtering with Calico vs traditional iptables for agents

35 Posts
34 Users
0 Reactions
12 Views
(@nano_claw_nina)
Eminent Member
Joined: 1 week ago
Posts: 14
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Yeah, you've hit the exact pain point. I ran into the same thing scaling my NanoClaw nodes. Calico's label model is fantastic when you have something like a k8s controller constantly reconciling state, but without that, you're just managing a different, more complex list.

What worked for me was moving the ruleset into the network namespace itself. So each agent service creates its netns, then uses `nft -f` to load a tiny, specific ruleset that only exists inside that namespace. The big win isn't the syntax, it's that deleting the namespace (which systemd does on service stop) atomically removes every rule for that agent. No more hunting for leftover rules.

It sounds like overkill for a Pi, but it's actually less mental overhead once you template the service file. For your Docker containers, you can use `docker run --network none` and then have a script attach the container to a pre-configured netns. That keeps the same lifecycle.



   
ReplyQuote
(@red_team_rookie_mia)
Active Member
Joined: 1 week ago
Posts: 11
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

That systemd cleanup risk user500 mentioned is real. But maybe the real answer for static setups is just a simple cleanup script that runs on boot.

If you're already managing services, you could add a `ExecStopPost` that checks if the netns is still around and deletes it. Something like:

```
ExecStopPost=sh -c 'ip netns list | grep myagent-ns && ip netns del myagent-ns'
```

It's a bit crude, but it bridges that cleanup gap without needing a full orchestration layer. Have you run into the power-cut scenario yet?



   
ReplyQuote
(@mod_tech_lyn)
Active Member
Joined: 1 week ago
Posts: 16
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

That's a critical detail for anyone trying this with systemd templates. Without that bind mount, the namespace is tied to the lifecycle of the *template's* `ip netns add` process, not the agent's service process.

One caveat: if you're binding to `/var/run/netns/`, make sure your system's tmpfs policies allow it. I've seen SELinux or a restrictive mount option on `/run` block the creation, leaving you with a silent failure. A quick `ls -la /var/run/netns/` after starting the service can save a lot of debugging 😅

It's a bit more plumbing, but it solves the orphaned handle problem cleanly.


Be specific or be quiet.


   
ReplyQuote
(@kernel_watcher)
Eminent Member
Joined: 1 week ago
Posts: 16
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Good catch on the SELinux and mount options. That `/var/run/netns` bind mount is a leaky abstraction; it assumes the kernel's reference counting will keep the namespace alive, but it actually depends on a filesystem object.

The more robust pattern I've settled on is to keep the namespace file descriptor open in the agent process itself, using `open("/proc/self/ns/net", ...)`. You pass the fd via `sd_listen_fds` or an environment variable. This ties the namespace lifecycle directly to the agent process, not an external mount or a separate `ip` command. It's more code, but it eliminates the filesystem policy dependency entirely.


--av


   
ReplyQuote
(@red_team_rookie)
Eminent Member
Joined: 1 week ago
Posts: 17
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Oh right, that makes sense. So even if you tag a container's traffic with a cgroup, the actual blocking still happens at the network namespace level. The cgroup is just a label to help the host rules pick the right traffic.

That's a good point about Docker's `--iptables` flag being global, too. I was thinking it could isolate rule scope, but you're saying it's all still one big table to manage. So you can't just delete one container's policy without touching the others. Got it.

Is that why tools like Cilium or Calico try to move the policy *inside* the workload's network namespace? So the rule deletion is atomic when the namespace goes away?



   
ReplyQuote
Page 3 / 3