How to build all buildable targets and test all testable targets

[alexeagle]

I want to "build and test all the things" on my CI.

bazel test ... is bad because it doesn't build a target unless some test depends on it. (probably surprising to new users)

bazel build ... && bazel test ... is bad because the first test doesn't start running until the slowest build target is built

At Google we give up on this, and use build_test (alluded to at https://docs.bazel.build/versions/master/be/functions.html#load) which is a special test target that always passes, but consumes the files from some given deps, forcing those deps to build, and thus making the test either fail with a build error, or pass. But I think this is bad because it's easy to forget to create a build_test for some target, and we shouldn't expect everyone to be ever-watchful for that gotcha.

Is there some other way I don't know of?

[iirina]

There's no way to do this (that I'm aware of), since building all buildable targets and testing all testable targets involve 2 different bazel commands with two different sets of targets. Unfortunately it's not how bazel was designed.

bazel build ... && bazel test ... is bad because the first test doesn't start running until the slowest build target is built

On the other hand, when bazel test starts, everything needed would have already been built.

CC: @aehlig @buchgr does BEP help in this case?

[buchgr]

@iirina wouldn't know how.

@alexeagle why don't you do bazel test ... && bazel build ...? The test invocation builds any build targets required by test targets, and the build invocation builds any remaining build targets (as the other ones are in cache).

[laszlocsomor]

bazel test $(bazel query //...) ?

[buchgr]

bazel test $(bazel query //...) ?

Doesn't this fail for non-test targets?

Nevermind. Nice!

[laszlocsomor]

No, you can "bazel test" a non-test target. Bazel will warn you about it, but will build it anyway.
The caveat with my proposal though is that it'll likely exhaust the command length limit.

[alexeagle]

@buchgr bazel test ... && bazel build ... might help in common cases, but you might still have a long-running non-test target like building your production server docker image. I could imagine a critical path becoming longer when it waits for all tests to finish first.

@laszlocsomor that's basically what I wish Bazel could do, without exhausting the command-line limit which seems like an easy pit to fall into.

Consider this a feature request?

[laszlocsomor]

@alexeagle : I'm afraid we don't have free cycles to work on this feature request right now.

[buchgr]

@laszlocsomor that's basically what I wish Bazel could do, without exhausting the command-line limit which seems like an easy pit to fall into.

bazel query //... | xargs bazel test should do the trick.

[alexeagle]

@buchgr not portable, but my main concern for this is CI where I control the environment. I can do this just for Linux/Mac builds to start with, and let Windows testing be slower.
Thanks!

[alexeagle]

It's a bit more tricky. We use tags=["manual"] to exclude tests from the test suite since it's not matched by //....
To restore that behavior, we also need --test_tag_filters=-manual on the bazel test command

[drigz]

bazel test ... is bad because it doesn't build a target unless some test depends on it. (probably surprising to new users)

Can you clarify this? If I have a basic workspace with a cc_binary and a cc_test, bazel test //... builds both and then runs the test. If I pass --build_tests_only then it behaves as you described.

[alexeagle]

If you didn't have a cc_test, it would build nothing. For example, you might build a binary that you distribute, but don't have an integration test for that binary. Then bazel test ... won't even try to build the binary, and you might break it.

…

On Thu, May 17, 2018 at 7:55 AM Rodrigo Queiro ***@***.***> wrote: bazel test ... is bad because it doesn't build a target unless some test depends on it. (probably surprising to new users) Can you clarify this? If I have a basic workspace with a cc_binary and a cc_test, bazel test //... builds both and then runs the test. If I pass --build_tests_only then it behaves as you described. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4257 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC5I9nqOWzdE_SVEGP4BVhfK8Kj3RpQks5tzY9VgaJpZM4Q6hz5> .