Rework breakpoint mechanism

[rageagainsthepc]

Now that we have added the required funtionality to KVMi we can rework the breakpoint API in order to incorporate SLAT switches. This would enable us to use guests with more than one vCPU. It would also increase performance in situations where the memory page that contains a breakpoint is read by the guest.

[phip1611]

This would enable us to use guests with more than one vCPU

Hey @rageagainsthepc,
should we interpret this as smartvmi not supporting multiple vCPUs at all at the moment? Because that is what we keep observing in our test setup.

We always get one of the following two errors:

• what(): setSingleStepCallback: Registering a second callback to the current VCPU is not allowed.
• Exception: storeOriginalValue: Breakpoint originalValue @ xxx is already an INT3 breakpoi

Can you outline what needs to be done to solve this? What is the quick and dirty solution? What is the solid solution? We might be able to work on this and contribute.

Thanks,
Philipp

[rageagainsthepc]

I am not entirely sure whether those errors are linked to a multiple vCPU setup but it's possible (and in the case of the second one very likely). Your assumptions are correct. Right now we only support a single vCPU. The problem is that we modify pages directly with INT3 writes. When multiple vCPUs hit the same breakpoint more or less simultaneously our breakpoint mechanism will inevitably break because multiple callbacks are trying to modify the same memory location. Usually the best way to solve this is by leveraging EPT:

1. copy the physical page
2. write the breakpoint on one of the identical pages
3. create a new EPT view
4. remap the original location so it points to the newly created page
5. modify permissions so that the page with the breakpoint is only X, but not RW
6. modify the page without the breakpoint so it is RW but not X
7. register an event to be notified for every thread context switch
8. switch EPTs accordingly for every thread context switch and also if one of the page generates a memory interrupt

If am not mistaken DRAKVUF does this in a similar way and there might also be a libvmi example if you are looking for an example on how to approach this.

If you are looking for a quick and dirty solution, I recommend using a single vCPU ;)