Live userspace process support for the Threads API

[ghost]

Building off of the work of #129, this PR (when completed) will implement and extend the Threads API outlined in #92 for live userspace processes. This will bring drgn one step closer to fulfilling the traditional role of a userspace debugger, most notably adding the ability to pause and resume threads at will during the execution of the program.

As this is still a work in progress, this PR is in draft mode.

[ghost]

@osandov Presently this is now a MVP of the feature. The core interface works well, has test cases, and is passing all of the existing tests. Due to the complexity and subtly of ptrace though (e.g. the 4 different types of ptrace-stop states, signal queuing, etc.), it's likely there are as-of-yet undiscovered edge-cases. Right now only x86 is supported, but adding other architectures should be relatively straightforward.

[osandov]

Sorry for the delay. I've been busy with other work, but this has been in the back of my head for awhile while I though about what the interface should look like. Thanks to your experimentation and MVP, I think we uncovered a few issues that we should consider:

1. ptrace hijacks threads exiting and receiving signals by default. I can imagine use cases for drgn that want this, and use cases that want the process to run undisturbed. So, I think that we should have interfaces to explicitly attach to and detach from individual threads, attach to all current and future threads, and detach from all threads. By default, we probably shouldn't attach to anything at the library level; we may or may not want the CLI to attach to everything by default.
2. ptrace is inherently event-based. This is pretty fundamental to debugging an external program: the program is going to asynchronously hit conditions that we'd like to react to. So, I think our API should be event-based as well. Specifically, I think we can have an interface that waits for the next event on a specific thread (example use case: I just sent a signal to a thread and I want to wait until that thread stops for that signal) and an interface that waits for the next event on any thread (example use case: I set a breakpoint and now I want the program to run until any thread hits it). It'd also be nice to have non-blocking versions of those (i.e., return and event immediately if it's available, otherwise bail).
3. ptrace supports various kinds of events, some of which are optional. We might want to have a way to enable/disable those optional ones, although I'm fine leaving that out for now. I think by default we want clone/fork/vfork, exec, and exit (which I believe is everything that ptrace supports other than seccomp).

This is a rough sketch of what I'm picturing:

class Program:
    def attach_all_threads(self) -> None:
        """
        Attach to all threads in this program as well as any new threads that
        are created.
        """
        ...

    def detach_all_threads(self) -> None:
        """Detach from all threads in this program."""
        ...

    def get_thread_event(self, block: bool = True) -> ThreadEvent:
        """
        Get the next event for any thread in this program. If *block* is
        ``True``, wait for the next event. If *block* is ``False``, raises an
        exception if no event is available (TODO: which one? Non-blocking
        sockets seem to return an OSError with errno set appropriately).
        """
        ...


class Thread:
    def attach(self) -> bool:
        """
        Attach to this thread.

        :return: ``True`` on success, ``False`` if the thread was already attached
        """
        ...

    def detach(self) -> bool:
        """
        Detach from this thread.

        :return: ``True`` on success, ``False`` if the thread was not attached
        """
        ...

    def interrupt(self) -> None:
        """
        """
        ...

    def continue(self) -> bool:
        """
        If this thread is currently stopped, resume it.

        :return: ``True`` on success, ``False`` if the thread was not stopped
        """
        ...

    def get_event(self, block: bool = True) -> ThreadEvent:
        """
        Get the next event for this thread.
        """
        ...

Then, we can have a dumb ThreadEvent object for different types of events. Maybe something like:

ThreadEvent = Union[ThreadEventSignal, ThreadEventExit, etc...]

class ThreadEventSignal:
    signal: int

class ThreadEventExit:
    status: int

etc...

The above (along with the equivalent libdrgn interfaces) is more or less what I'd consider the MVP for this design. There are obviously lots of finer details to consider, as well as complications with ptrace that I may not have considered.

There are some "extras" that we should implement eventually, possibly in follow-up PRs:

• We might still want a shortcut Thread.pause(), which I think would basically be:

def pause(self):
    self.interrupt()
    while True:
        event = self.get_event()
        if isinstance(event, ThreadEventStop):
            return
        # TODO: should we have a distinction between "stopping" events that
        # require a continue and "non-stopping" events?
        self.continue()  

• ptrace allows suppressing signals that the program received. Should we have a suppress_signal: bool = False parameter to continue()?
• How would we expose PTRACE_SYSCALL and PTRACE_SINGLESTEP? More parameters for continue()?

This is definitely larger in scope than the basic pause and resume API from before, but I'm hoping that it makes drgn useful for a much more diverse set of tools. I'd love to hear your thoughts.

[osandov]

I wrote this project up as an issue, so I'm closing this one. Thanks again for getting this started!