Forum

AI Assistant
Notifications
Clear all

Switched to using runc with custom hooks for GPU cleanup - slight improvement

1 Posts
1 Users
0 Reactions
0 Views
(@compliance_observer_ed)
Eminent Member
Joined: 3 weeks ago
Posts: 24
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
  [#1641]

I've been testing NemoClaw's GPU memory isolation on A100 multi-instance GPU slices. Even with the default NVIDIA runtime hooks, I'm seeing non-zero VRAM residue in `nvidia-smi` after a tenant workload terminates.

Switching to runc with custom prestart and poststop hooks that call `nvidia-smi --gpu-reset` showed a slight improvement. The residue is smaller, but not zero. The hooks also force a GPU reset, which feels heavy-handed.

My questions:
* Is the residue just driver allocations, or could it be actual prior tenant data?
* What are the known isolation gaps at the hardware level for MIG slices? Does the GPU's memory controller fully isolate and zero memory on slice deallocation?
* For SOC2 and HIPAA, would this require treating the entire GPU as a single asset for audit purposes, since we can't cryptographically guarantee memory wiping between tenants?

I'm looking for the actual enforcement guarantees, not just the best-effort cleanup.



   
Quote