Run target in debugger

[cbarrete]

A common workflow for me is to always run binaries (both applications and tests) under a debugger, so that I can immediately investigate what is going on (as a side note, this is very common among Visual Studio users, e.g. in the games industry).

This is very useful to save time and to catch rare bugs, but it empirically only really works if it's convenient enough. Unfortunately, manually typing out buck-out paths or hacking around the output of buck2 build --show-output is hardly convenient.

Would you be open to adding support for this use-case via a new command/flag (e.g. buck2 debug or buck2 run --debug)? If so, I'd be happy to discuss the actual design and/or work on the implementation.

[zjturner]

Not a core developer, but another constant debugger user.

I'm just curious how you envision this working. buck2 would have to know which debugger you want to use, and it would have to know how to find it on your system. Maybe you want to launch it in Visual Studio Code, or in Visual Studio, or in WinDbg for example.

If you're a Visual Studio user, then perhaps what you really want is IDE integration, because then you just hit F5 and it works.

It seems like this is something that would be pretty trivial to implement outside of buck2. You could have a python script (for example) that ran buck2 targets <foo> --show-output, parsed that, and then ran whatever logic you wanted to spawn the program in your debugger.

[thoughtpolice]

FWIW, you can just use something like genrule() or sh_binary() in order to write a script that can be executed with buck2 run, so you can just put all the debugging logic under that. For example, buck2 run path/to:target could launch it under a debugger by default, you could set an environment variable DEBUG=1, etc.

If you need something reusable, you can write a macro for it (e.g. write your own cxx_binary that writes both a debugging script out and calls the real cxx_binary)

See also #86; assuming #86 (comment) was implemented, you'd be able to also do this in BXL trivially, and then map the long-and-verbose BXL command back to a quick shorthand such as buck2 my-debug ..., which would remove the need for genrule-based things.

[cbarrete]

Thanks for the comments!

I'm just curious how you envision this working.

I'm not totally sure yet, but to me important points include:

1. Good integration with buck2: have all buck2 flags automatically apply, all binary targets be available, be as convenient to run as running the binary bare (I might even go as far as to say that running a binary bare is a special/subcase of running it in general), etc.
2. Be customizable by regular users (not just buck2 experts): having to maintain local .bzl/BUILD files and set them up on each clone just to setup debugging means that most people won't as easily reach out for it. On the other hand, I don't think that you'd want each repo to maintain a lot of common boilerplate, so perhaps a/the prelude would be a good place for that? I don't know yet.

If you're a Visual Studio user, then perhaps what you really want is IDE integration, because then you just hit F5 and it works.

I'm not (anymore), I personally just want to get gdb running in a terminal. I agree that debugging from within an IDE should be solved by whatver the IDE integration is, but I don't know if that also applies to e.g. standalone graphical debuggers, which I would expect not to be build system aware.

It seems like this is something that would be pretty trivial to implement outside of buck2.

Sure, but the big revelation for me when working in a debugger-first environment was how much of an impact there is that it is the path of least resistance. Then again, I would understand if this was out of scope for a build system.

@thoughtpolice: very interesting suggestions, though I don't understand the implications as I don't know BXL yet. Some initial concerns that you might have answers to:

1. would this be pushing a bunch of complexity to each project? I don't know how much boilerplate would be involved; and it's not clear to me where the line between upstream buck2 and each project should be in implementing this.
2. Would it be reasonably easy to get seamless integration (reusing the same flags, getting shell completion, etc.)? It looks like it from the examples you've linked, but they also look a little bit like magic to me at the moment (and I'd like the outcome to be accessible to anyone using buck2)

[cbarrete]

By the way @zjturner, since you've mentioned that you're a constant debugger user, what is your current workflow with buck2?

Say that a buck2 test ... fails and you want to debug the specific test that failed, what do you do?
Something like *copy the target of the test that failed*, buck2 targets --show-output *paste*, *copy the path of the binary*, gdb *paste*, then adjust the path that you've just pasted so that it's relative to your current directory?

That's what I've been doing so far and it's... not great; so maybe I'm just missing something.

[zjturner]

TBH I haven't gotten that far yet. I'm porting a very large codebase, and it's going to be a long effort before I get to the point that I can even think about debugging some targets.

That said, it will probably be some kind of external solution. I don't think it usually makes sense to integrate debugging with the build system. The build system should just build, but it should also provide the necessary discoverability and introspection necessary to allow external integration of the build with debuggers.

That probably means either IDE integration or a shell / python script that runs buck2 targets --show-output behind the scenes and launches the executable.

IDE integration also applies to GDB or LLDB too. You can alias a command in GDB to run a GDB Extension script, so from within your GDB terminal you can write something like buck2-debug //path/to/target -- args and it will run a script which queries for the output path, and loads it in GDB with the specified args.

[benbrittain]

don't think it usually makes sense to integrate debugging with the build system.

buck2 already has buck2 run as a first class citizen. a buck2 debug is minimally different, just a different runtime basically.

I'm fairly supportive of this idea. Telling a user to dump their last run command and slip a gdb in there is pretty sub-optimal.

[zjturner]

buck2 already has buck2 run as a first class citizen. a buck2 debug is minimally different, just a different runtime basically.

I think it’s extremely different, for all of the reasons described earlier. But in particular, which debugger and what is the interface for talking to it?

there is one way to run a process, by calling exec or CreateProcess. No configuration is required because you are interfacing directly with the operating system.

if you’re going to run a debugger, user configuration is necessary because debugger is third party software buck2 has no control over. Maybe it's not in PATH. Maybe it's not GDB (GDB is only really a single-platform thing, and buck2 supports multiple platforms). Maybe it's a forked / internal version of a debugger that has some differences in the way it has to be launched (very common at large companies). Lots of other reasons why talking to a debugger isn't really a one-size-fits-all thing.

I actually think it’s optimal to do this in the reverse direction (teach debugger about build system rather than teach build system about debugger). It’s very simple (couple of line debugger extension script) and supports every possible use case.

Furthermore, with the exception of Linux / GDB, most people are using graphical debuggers that make more sense to integrate with from the IDE side. So it feels like the idea of integrating debugging as a first class thing in buck2 is very very specific to Linux / GDB, which rules it out by virtue of not being general purpose enough to warrant first class inclusion in the core runtime.

Telling a user to dump their last run command and slip a gdb in there is pretty sub-optimal.

The approach I suggested in my earlier message is quite a bit more streamlined than this. You don't run a command and slip gdb into it. You instead write buck2-debug //path/to/target in your debugger and it just works.

[cbarrete]

The configuration that you're describing could be exposed by the toolchain, and there doesn't need to be a default.

I also don't really see how it is so much more complicated than buck2 run or buck2 test. You could also argue that IDE integration should be the way to go about those. My understanding is also that there is per-toolchain configuration of those via RunInfo already, to manage arguments among other things, it's not just execing a path.

if you’re going to run a debugger, user configuration is necessary because debugger is third party software buck2 has no control over. Maybe it's not in PATH. Maybe it's not GDB (GDB is only really a single-platform thing, and buck2 supports multiple platforms). Maybe it's a forked / internal version of a debugger that has some differences in the way it has to be launched (very common at large companies). Lots of other reasons why talking to a debugger isn't really a one-size-fits-all thing.

Which sounds exactly like the toolchains that you'd have to setup in buck2 as well anyway. I get that buck2 is not a debugger configuration system, but this could all easily be passed e.g. to a toolchain as a debugger parameter or something similar.

Furthermore, with the exception of Linux / GDB, most people are using graphical debuggers that make more sense to integrate with from the IDE side.

And people using an IDE are more likely to build/run/test from within it, yet buck2 exposes user-facing commands for that.

So it feels like the idea of integrating debugging as a first class thing in buck2 is very very specific to Linux / GDB

I don't have statistics on debugger popularity/usage, but debuggers outside of GDB and lldb also work in a standalone way (e.g. RemedyBG, DDD and I'm sure more). The fact that not many build systems integrate with them is a mistake to me. Also, oher build systems don't have native, convenient way of running specific targets or tests, whereas buck2 does, and for good reason. On the other hand, it really feels like a missing piece to be able to conveniently build, run, test your code, but just not debug it.

The approach I suggested in my earlier message is quite a bit more streamlined than this. You don't run a command and slip gdb into it. You instead write buck2-debug //path/to/target in your debugger and it just works.

I essentially do this at work on a CMake codebase and it's annoying that it's just a local hack. Even if I check it in, it wouldn't be part of other people's PATH and would just be a random script that some people might want to use, as opposed to something well integrated and fairly standard. I'll resort to scripting for myself if I have to, but scripting around an otherwise consistent and accessible system for something common is weird to me.

Anecdotally, I've suggested a colleague to run his code in a debugger to quickly figure out where a segfault was happening earlier today, but it was too much of a hassle for them to setup a debugger, where I wish they could have just cmake debug my_target.

[alexlian]

A common workflow for me is to always run binaries (both applications and tests) under a debugger, so that I can immediately investigate what is going on (as a side note, this is very common among Visual Studio users, e.g. in the games industry).

Back at the studio I was at, we solved this by adding a --dbgbreak flag to the binaries that would invoke DbgBreak() and let the Windows system prompt the attaching of the debugger of choice. The benefits include any-time-any-where debug attachment ease, along with skipping slow initialization points depending on how you place your dbgbreak call. The downside of course is if you needed to debug those early phases.

Internal to Meta, the debugging tooling prefixes buck2, however I'm naive to its implementation. Perhaps another team member might explain. Basically it's a debug_wrapper buck2 test <target> invocation.