Skip to content

Forum

AI Assistant
Notifications
Clear all

Am I overthinking it by wanting to run NIM on a separate, isolated VLAN?

1 Posts
1 Users
0 Reactions
0 Views
(@hardening_hector)
Active Member
Joined: 1 week ago
Posts: 9
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
  [#1173]

Deploying NIM with default configs is asking for trouble. It's a complex inference service, often with elevated privileges and open network ports. Treating it like any other app on your main VLAN is naive.

Key reasons for isolation:
* Model files are high-value targets.
* The container often runs with `--privileged` or excessive caps for GPU access.
* Default NIM config binds to 0.0.0.0.

You're not overthinking it. A separate VLAN with strict firewall rules is the bare minimum. Better yet, a dedicated physical host or a VM with no other workloads.

Basic VLAN tagging and firewall example for Linux bridge:
```bash
# VLAN 50 for NIM hosts
ip link add link br0 name br0.50 type vlan id 50
ip addr add 10.0.50.1/24 dev br0.50

# Isolate: allow only management SSH and specific NIM port from jump box
iptables -A FORWARD -i br0.50 -o br0 -s 10.0.50.10 -d 10.10.10.5 -p tcp --dport 22 -j ACCEPT
iptables -A FORWARD -i br0.50 -o br0 -s 10.0.50.10 -d 10.10.10.5 -p tcp --dport 8000 -j ACCEPT
iptables -A FORWARD -i br0.50 -o br0 -j DROP
```

Without this, a compromise could pivot to your entire lab network.

--harden


Drop the --privileged flag.


   
Quote