UI request: a shorthand for 'buck bxl' invocations (or: arbitrary 'buck foobar' commands)

[thoughtpolice]

Here's a nifty one-liner I put behind the name bxl:

exec buck2 bxl "bxl//top.bxl:$1" -- "''${@:2}"

This requires you to:

• set bxl = path/to/some/dir in your root .buckconfig file [repository] stanza, and then
• export a bunch of top level bxl() definitions from bxl//top.bxl which can be invoked

Then you can just run the command bxl hello --some_arg 123 in order to run an exported bxl() definition named hello. Pretty nice! And because the command invocation uses the bxl// cell to locate top.bxl, the bxl command can work in any repository that defines that cell, and the files can be located anywhere.

---

So my question is: could something like this be supported in a better way? The reason is pretty simple: it's just a lot easier to type and remember!

Perhaps the best prior art here I can think of is git, which allows git foobar to invoke the separate git-foobar binary, assuming it's in $PATH and has an executable bit set. We don't need to copy this exactly 1-to-1, but it's a generalized solution to this problem, and in fact it's arguable that being able to do a buck2 foobar subcommand is useful for the same reasons. So maybe that's a better place to start, and the bxl scripts could be a more specialized case of this.

[ndmitchell]

It's great that Bxl is turning out to be so useful. Having top.bxl be hardcoded to being special seems unfortunate, and I'm not a huge fan of looking up things on the $PATH. But you could imagine adding entries to .buckconfig:

[commands]
bxlrun = "buck2 bxl bxl//top.bxl:$1 $*"

Or something of that form - so you can essentially introduce top-level command aliases to buck2. I could imagine defining fasttargets as an alias for targets --streaming --keep-going --no-cache --inputs (which is a command I do quite frequently).

[thoughtpolice]

Yes, I think that something like that is the right way forward, but I didn't want to get too far ahead of myself!

A big advantage of having it in the .buckconfig file rather than being implicit is that it's per project. In git, to invoke git foobar it needs to be in $PATH while buck bxlrun (in your example) is localized to the repository in an easy to understand form that can be immediately looked up and referenced. That would make discoverability much better. (Another advantage is that you can rule out name conflicts between commands, but in practice in the git ecosystem this never seems to have been a problem after many years.)

[thoughtpolice]

Here's something I just thought of: if a command is exposed as a buck2 subcommand, what about UX considerations like the help system? Like, if the set of commands is configured in .buckconfig somehow, in whatever way — I think it might be reasonable to expect the UX to reflect that. So buck2 help should also show bxlrun too, right? And it's arguments if it takes some.

I think this is highlighting is that the [commands] example before might better be thought of as an alias. But a real "subcommand" might be thought of as being semantically different and having a different set of UX expectations. Both are useful, I think.

I point this out because I think part of the build system is having a great UX. git's way of doing things doesn't really have any way of enforcing this, which is arguably fine, but there's no reason to copy that. So thinking this through a little, it might be better to do it "the harder way", which would probably be to support a more programmatic design.

---

Here's a more worked example of what I'm thinking of. Let's say .buckconfig looks like this:

[repositories]
bxl = path/to/bxl/dir

[aliases]
fasttargets = "targets --streaming --keep-going --no-cache --inputs $*"
brun-less = "bxl bxl//$1.bxl:$1 -- ${*:2}"

[commands]
brun-more = bxl//brun.bxl

Then, path/to/bxl/dir/brun.bxl could be:

"""A command that runs bxl scripts, conveniently"""

def _impl(ctx):
    # NOTE: execute `buck2 bxl bxl//${target}.bxl:${target} -- ...` similar to the alias

_args = {
  "target": cli_args.option(cli_args.target_label())
}

main = bxl(impl = _impl, args = _args) # for `[commands]` stanza
brun = bxl(impl = _impl, args = _args) # for `brun-less` to work

All of the following commands now work:

• buck2 fasttargets //..., which works "as expected"
• buck2 brun-less foo --cool_beans 123, which similarly works as expected; to be clear it effectively translated to buck2 bxl bxl//foo.bxl:foo -- --cool_beans 123, assuming bxl//foo.bxl exists and has a top-level foo = bxl() export.
• buck2 brun-more --target foo -- --cool_beans 123 would also work assuming brun.bxl was implemented appropriately and just executed the expected buck commands
• buck2 brun-less brun --target foo -- --cool_beans 123, which runs the alias to run the command to run another command!

This is a contrived example but I think you see what I'm getting at. The semantics are something like:

• An [alias] stanza entry refers to a simple alias for some other existing buck2 command
• A [command] stanza refers to a path expression (not a target!), identifying a .bxl file, which can expose the proper API to be loaded, queried, etc.

Some notes:

• Aliases do not mention buck2 as the command itself; it's always implied as the first "word" of the command to invoke — this keeps aliases more focused and specific as shorthands for existing buck2 features, while commands can be "do any arbitrary thing and invoke arbitrary executables"; but this is just a minor nit/aesthetic point; feel free to disagree.
• Note the usage of ${*:2} in the [aliases], which is a kind of "array slice" similar to the bash version, because we need to split the parameters appropriately. Something like this needs to be worked out.
• There is no reason that the BXL API can't support multiple subcommands of its own commands either, but it would need to be worked out. This is why I left the details mostly abstract
  • So then we can have buck2 foobar subcmd1 --flag2 true --flag3 false defined by foobar.bxl or whatever

At this point if this could work, the rest mostly falls into place. buck2 help could then show more detailed programmatic output for all [command] stanza entries, and it could even show a "The following aliases are also currently defined:" as well in the help output for the simpler case, too.

Final notes:

• The [command] part is reasonably heavy enough to probably require some kind of RFC
  • Mostly, a BXL API will need to be thought up.
  • It's probably definitely a bunch of work, I bet.
• The [aliases] part is probably a much simpler, easier win
  • The slicing syntax does need to be figured out
• None of this is required, because the workarounds can be done in my case, but I think it's nice. :)

[ndmitchell]

Cool, I'll raise this and #85 with the larger team internally and encourage people to comment here.

[benbrittain]

Was there any commentary from the internal team on this feature? As discussed in #263 we are running into some real world use cases where this could be a nice addition.

[cbarrete]

@ndmitchell Has anything come out of those internal discussions? Would you accept a patch that would add support for aliases, at least as a first step?