Add bazel_git and protobuf-java packages

[divanorama]

bazel_git is built from fixed recent git sources.

There are several bootstrap options
https://docs.bazel.build/versions/master/install-compile-source.html

• use release dist sources and compile without bazel installed
• use git sources and compile with some bazel version
• use git sources and supply extra bootstrap jars&tools

We could compile with stable versions but this can cause more
expensive rebuilds and we can't expand same compile method to
all bazel versions in nixpkgs.

Few extra changes that were needed to make bootstrap compilation work:

• provide protobuf-java package
• change update-srcDeps to depend on "bazel query" output as newer
  versions have more complex dependencies setup with quite some Starlark
  involved
• don't rewrite src-deps.json on update-srcDeps.py errors

Notes:

• bazel version is set to 3.9999, which makes sense because there's
  no single version yet, similar to say 3.7.0-SNAPSHOT but keeping
  the numeric format. This may affect .bazelversion or version_check()
• hopefully we can bump bazel_3 to 3.8.0 once that release is
  available. Versions 3.4 to 3.7.0 have issues with dependencies
  without sha256 or even @master dependencies
• we can also switch release versions compilation to same bootstrap
  process not relying on dist archives. Potentially may introduce
  differences from binary releases, but we'd be testing full bootstrap
  and rely on git sources.
• "bazel version" reports
  Build label: 3.9999 ~=3.7.0 (@259930228cfe505c015055b0cec62a406b4881910)
  bazel_3 output is
  Build label: 3.3.1- (@Non-Git)

Motivation for this change

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[divanorama]

looks like this one fails in sandbox, need to somehow put maven dependencies in

[divanorama]

also fails in sandbox

Building Bazel from scratch......
Building Bazel with Bazel.
...
ERROR: no such package '@bazel_toolchains//rules': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/releases/download/3.1.0/bazel-toolchains-3.1.0.tar.gz, https://github.com/bazelbuild/bazel-toolchains/releases/download/3.1.0/bazel-toolchains-3.1.0.tar.gz] to /build/bazel_AFAZt0Fc/out/external/bazel_toolchains/temp4351923723686256938/bazel-toolchains-3.1.0.tar.gz: Unknown host: github.com

probably 2nd phase doesn't use our distdir bazelrc setting because bazel-toolchains-3.1.0.tar.gz is present in src-deps.json

[Profpatsch]

Bazel has a pretty tight release cycle (every 2 months or so), is there a particular reason we want a master version? I don’t see it being updated regularly to be honest, and if you need it for your downstream project it should be possible to just copy these expressions there.

[divanorama]

I don't have a strong need for git version, but we'll anyway need some of those changes to build 3.8.0+ once it gets released. Can keep this in my fork and prepare 3.8.0 PR later though

[Profpatsch]

Yes, that would be nice! Thanks.

…

On Thu, Oct 22, 2020 at 2:32 PM divanorama ***@***.***> wrote: I don't have a strong need for git version, but we'll anyway need some of those changes to build 3.8.0+ once it gets released. Can keep this in my fork and prepare 3.8.0 PR later though — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#101292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYB5ZVZFJFBAUAH7WYKQOTSMAQ6TANCNFSM4S2MCCCA> .