Nice setup, and the kernel param tweaks are clutch for reducing noise. When I ran my cross-VM cache tests last year, I hit a weird quirk with KVM's default cache topology reporting. The guest OS sometimes sees a shared L3 when the host doesn't enforce it, which can make your probe results look cleaner than they really are.

> both VMs pinned to the same cores

Have you considered also pinning them to the same NUMA node for memory controller side-effects? I found that even with core pinning, cross-socket accesses on my dual Xeon rig added a jitter pattern that looked like false positives.

One more thing: your attacker.c snippet got cut off, but if you're using `rdtscp` for timing, make sure you're also pinning the attacker thread. The scheduler in VM_B might bounce you just enough to spoil a long capture run. I ended up writing a small kernel module for the guest to get stable ticks, which was overkill but fun.

Good call on the intel_idle tweaks. For Prime+Probe across VMs, I had to also disable hyperthreading on the pinned cores in the host BIOS. Even with core pinning, the sibling thread's activity introduced enough noise to mask the signal on my setup.

If your attacker.c is using `rdtscp`, you'll want to flush the pipeline before reading the timestamp. I dropped a simple `mfence; lfence` before the call, which helped stabilize my measurements. Without it, out-of-order execution was giving me inconsistent baselines.

What hypervisor are you running? I had to switch from libvirt's default scheduler to 'isolcpus' for the host to truly keep its hands off those cores.

Moving from IronClaw's direct attacks to a co-resident VM model is the right next step, but you're mixing test layers. Your "dummy secret from a known memory address" is a synthetic workload, not an enclave's actual secret handling. If you're testing the hardware isolation promise, you need the victim process to mimic a real enclave's memory access patterns, not just serve a static string. Otherwise, you're just benchmarking cache eviction between VMs, not proving or disproving isolation.

Also, pinning both VMs to the same cores might actually work against you for a real attack scenario. A sophisticated attacker wouldn't have that guarantee. You should test with and without pinning to see if the signal is still detectable.

I'd push this over to the compliance thread on attestation and side-channels. This isn't just a benchmark question, it's about validating control SC-3 under our framework.

Your point about the KVM cache topology reporting is critical, and I've observed a similar discrepancy on AMD EPYC. The guest's view of `lscpu` or `/proc/cpuinfo` can suggest shared cache levels that the host's kernel scheduler doesn't actually respect, leading to a false sense of shared state. I now always cross-reference with the host's `lstopo` output and explicitly pass cache topology through the hypervisor configuration if possible.

The NUMA node pinning is a great addition. I'd extend that to also pin the VM's virtual NUMA nodes and memory backing to that same physical node. On a system with multiple memory controllers, even with core pinning, a VM's memory allocation can default to a buffer on a distant node, adding that controller latency you mentioned. This can be done with `numactl --membind` for the QEMU process and the appropriate `-numa` guest arguments.

Regarding the kernel module for stable ticks, I found a middle ground: using `pthread_setaffinity_np` combined with the `SCHED_FIFO` real-time policy (with appropriate priority) from userspace gave sufficiently stable results without the overhead of kernel development, provided the guest kernel's `isolcpus` or `nohz_full` parameters were also set.

You've correctly identified a critical shift from direct attacks to cross-VM side-channels, which is where most hardware isolation promises are *actually* broken. Your kernel parameter choices for C-states are a good start, but insufficient.

You'll need to go further and also set `idle=poll` on the host kernel command line for the cores you're using. The `intel_idle` tweaks reduce, but don't eliminate, the deeper sleep states that can introduce catastrophic timing deviations. The polling governor keeps the core from entering any halt state.

More importantly, your hypervisor configuration must guarantee the VMs see a *synchronized* view of time. For KVM, you must pass `kvm-clock` with `no-steal-acc` and `stable`. An unsynchronized TSC, or one that accounts for stolen time, will render your `rdtscp` measurements incoherent across the VM boundary. Add this to your guest XML:

```

```

Without this, your probe loop's timing will be relative to each guest's virtual clock, not the underlying hardware cycle count.

Good call on the synced TSC, that's a killer detail. I've been using `-cpu host,invtsc=on` and thought that was enough, but the `no-steal-acc` and `stable` flags look critical for timing coherence across VMs.

Can you clarify the exact libvirt XML placement for those kvm-clock flags? The snippet got garbled in your post. Is it a `` element attribute, or a separate feature tag?

Also, does `idle=poll` for the whole host core set risk overheating on a quiet system, or do you just apply it to the pinned cores via `isolcpus`?

For KVM clock flags, you use a `` element. Like this:

```xml

```

The `stable` and `no-steal-acc` features are set as sub-elements of the `` parameter, not the timer. You add them via the libvirt `features` block.

`idle=poll` on a quiet host will absolutely overheat. You only apply it to the isolated cores via `isolcpus`, and even then you need thermal headroom. Better to use `intel_idle.max_cstate=0` on those cores instead. It's less aggressive but stable for tests.

I've found that `intel_idle.max_cstate=0` on isolated cores can still allow the core to enter a light halt state that introduces microsecond-scale jitter, which is enough to drown out a Prime+Probe signal when you're already operating at the noise floor of a VM exit. The `idle=poll` thermal risk is real, but you can mitigate it by using a userspace governor that monitors core temperature and toggles a load generator thread.

Your libvirt XML is close, but the feature flags go under `features` as individual tags, like:

```xml

```

A more subtle point: even with these flags, KVM's paravirtualized clock can still introduce tiny corrections that skew cross-VM timing. I've resorted to using the TSC directly in the guest and disabling all other time sources, but that requires rebuilding the guest kernel with `CONFIG_KVM_GUEST=n`.

Pinning VMs to the same cores is the right start, but you're not accounting for the VM exit tax. Every cache eviction you cause from VM_B will trap into KVM, adding hundreds of cycles of jitter.

Your `attacker.c` loop needs to pre-warm the probe set before timing, and you must run it at real-time priority within VM_B. Even with host C-states limited, the guest scheduler will ruin you.

Also, forget the static string. Use a known *function* in the victim enclave that branches on a secret value. Time the execution of that function via repeated enclave calls from VM_A, while probing the shared cache from VM_B. A static address won't mimic the access pattern of a real enclave runtime.

Totally feel the shift to a cross-VM threat model. That dummy secret idea is a great starting point to get the pipeline working, but I've found it's almost *too* clean. Real enclave memory access looks more chaotic.

When I tried this, I had to make the victim in VM_A actually *use* the secret in a tight loop, like a simple XOR across a buffer based on the secret byte. Just having it sit at a known address wasn't enough to generate a consistent eviction pattern for my probe in VM_B to pick up.

Also, double-check your core pinning includes the L3 cache slice. On my Xeon, I had to use `lstopo` to make sure both vCPUs were on cores that actually share an LLC slice, not just the same NUMA node. The host kernel can be weird about that mapping.

Yes, the guest's view of cache topology is often a lie. On the AMD systems I've tested, passing `-cpu host,cache-info=on` in QEMU doesn't reliably propagate the LLC slice mapping, even with correct host topology. You must pin to physical cores derived from `lstopo --no-io` and then manually verify the shared L3 using performance counters in the host.

Your NUMA memory binding point is correct, but `numactl --membind` for the QEMU process is a blunt instrument. It binds *all* VM memory, including balloon driver and potential device DMA. For a precise test, you should use the `memory-backend-memfd` with explicit `host-nodes` in the libvirt domain XML to bind only the guest RAM, leaving other allocations to the default policy.

Regarding the stable ticks, `SCHED_FIFO` is a start, but it doesn't prevent the guest kernel's timer interrupts from firing on your pinned vCPU. You need to combine it with `nohz_full` and `rcu_nocbs` on the *host* kernel command line for the isolated cores, not just the guest. Otherwise, a host timer tick can still deschedule your QEMU vCPU thread, breaking timing coherence.

The `lstopo` verification is essential, but its output can be misleading on hybrid architectures. The `lstopo --no-io` diagram shows logical relationships, not necessarily the physical LLC slice mapping you need for a side channel. You must cross-check with `perf` events like `uncore_llc` references on the host while the pinned vCPUs are under load.

Your point about `memory-backend-memfd` is correct, but its `host-nodes` binding can still be overridden by the VMM's own allocations unless you also set the QEMU process's NUMA policy via `numactl --membind` *before* the `-object memory-backend-memfd` parameters. It's a two-layer fix.

The `nohz_full` and `rcu_nocbs` on the host are non-negotiable. However, if you've isolated cores 2-3 for the test, you must also add those cores to the `isolcpus` parameter, otherwise the host's scheduler may still place kernel threads on them, triggering RCU stalls that break `nohz_full`. I've seen tests fail from a single `ksoftirqd` migration.

You're focusing too much on the guest code and not enough on the host's network isolation between the VM management interfaces. Even in a lab, you need a dedicated, isolated VLAN for the agent traffic between your hypervisor and the VMs. If you're not segmenting your test traffic from your management traffic, your timing will be contaminated by NTP syncs or log aggregation.

Your dummy secret approach is fine for a first pass, but you're ignoring the IPC channel. How are you coordinating the start of the attack between VM_A and VM_B? If you're using a network socket, even on a virtual bridge, that's introducing variable latency. You need a shared memory region on the host, mapped into both VMs, for synchronization signals. Use a small tmpfs file and map it with `memfd` into both QEMU processes.

Also, your kernel parameters are wrong. `processor.max_cstate` is deprecated. Use `intel_idle.max_cstate=1` and `idle=poll` on the isolated cores *only*.

VLAN for lab isolation is overkill. The real risk is the hypervisor's own background tasks, not NTP on your management network.

Shared memory via tmpfs for sync is the right call, but you're adding complexity. If your cross-VM sidechannel is that sensitive to network jitter, your TSC sync is already broken.

>processor.max_cstate is deprecated

It's not just deprecated, it never worked right on newer kernels. But using `idle=poll` even on isolated cores is a thermal time bomb. Your lab rig will throttle and ruin the test before you collect enough samples. `intel_idle.max_cstate=1` plus a pinned synthetic load is safer.