Skip to content

Forum

AI Assistant
Notifications
Clear all

My results after testing network throughput between two agent microVMs.

1 Posts
1 Users
0 Reactions
0 Views
(@selfhost_raj)
Eminent Member
Joined: 2 weeks ago
Posts: 27
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
  [#1456]

Alright, so I finally got around to my weekend project: benchmarking the network throughput between two agent microVMs. I've been moving more critical logic into isolated microVMs (using Firecracker) instead of plain Docker containers, and everyone always asks about the performance hit. Well, here's some real data from my homelab.

My setup:
* Two Firecracker microVMs, each running a minimal Alpine image.
* Agents inside are simple Go services that echo network traffic for the test.
* Host system is an older Xeon with 64GB RAM, running the microVMs via `firecracker-containerd`.
* For comparison, I ran the same test between two regular Docker containers on the same host.

The results were... interesting. Direct TCP throughput between the microVMs averaged around **1.2 Gbps**. The same test between plain containers? A full **9.8 Gbps** (nearly saturating the 10G link). So yeah, there's a significant cost – roughly an order of magnitude in raw throughput.

But before you dismiss microVMs, consider the gotchas:
* The overhead isn't just from Firecracker; it's largely from the virtual network stack. I was using a simple bridge setup.
* For most of my agent workloads (think API calls, small data fetches), latency and connection overhead matter more than raw bandwidth. The microVM penalty there was under 0.5ms.
* This is the trade-off for that real hardware-level isolation. A container breakout gets you the host; a microVM breakout gets you... a tiny, stripped-down kernel.

Has anyone else run similar tests? I'm curious if tweaking the virtio-net queues or trying a different backing network method (like a dedicated tap) could close the gap a bit. I'm planning to test with gVisor's `sock` mode next, which promises a lighter network path.

~ Raj


Selfhosted since 2004


   
Quote