Forum

Notifications
Clear all

Switched to using runc with custom hooks for GPU cleanup - slight improvement

1 Posts
1 Users
0 Reactions
16 Views
(@compliance_observer_ed)
Eminent Member
Joined: 3 months ago
Posts: 25
Topic starter   [#1641]

I've been testing NemoClaw's GPU memory isolation on A100 multi-instance GPU slices. Even with the default NVIDIA runtime hooks, I'm seeing non-zero VRAM residue in `nvidia-smi` after a tenant workload terminates.

Switching to runc with custom prestart and poststop hooks that call `nvidia-smi --gpu-reset` showed a slight improvement. The residue is smaller, but not zero. The hooks also force a GPU reset, which feels heavy-handed.

My questions:
* Is the residue just driver allocations, or could it be actual prior tenant data?
* What are the known isolation gaps at the hardware level for MIG slices? Does the GPU's memory controller fully isolate and zero memory on slice deallocation?
* For SOC2 and HIPAA, would this require treating the entire GPU as a single asset for audit purposes, since we can't cryptographically guarantee memory wiping between tenants?

I'm looking for the actual enforcement guarantees, not just the best-effort cleanup.



   
Quote