[bazel] lowRISC/misc-linters does not cache Python wheels

[timothytrippel]

The lowRISC/misc-linters repository currently makes use of the default rules_python repository to register a hermetic Python toolchain and packages. Like #12091, the Python wheels in this dependency must be cached so that this (opentitan) repo can be built in an air-gapped environment. To accomplish this:

Addressed in: lowRISC/misc-linters:#28

• the lowRISC/misc-linters repository must be updated to
  • use the patched rules_python repository (that allows passing the pip_install rule a local directory to look for packages before going to the network), and
  • cache the Python wheels downloaded by the pip_install rule (similar to [bazel] add pip_wheel repository rule #12091)

Addressed in: #12562

• the commit and sha of the lowRISC/misc-linters dependency (in this Bazel workspace) must be updated to include the above updates

Addressed in: TBD

• determine what is causing the lint rules to be a dep of the chip-level test (opentitan_functest) targets

[rswarbrick]

As a meta-question: do we care? It seems a bit bizarre to be hot on running a licence check linting flow in an air-gapped environment! Can't we just not bother running it?

[timothytrippel]

That's a great question, and one I asked myself before I endeavored down this path, and the answer I still believe is that we shouldn't need to! However, reality set in, and I had to side line my idealist approach in the interim.

Specifically when I run a bazel build --distdir=$BAZEL_DISTDIR --repository_cache=$BAZEL_CACHE //sw/device/tests:aes_smoketest on the air-gapped machine, it hangs, trying to fetch the @lowrisc_linter repository, see below screen shot:

But this was puzzling to me because the bazel documentation clearly states "By default, external dependencies are fetched as needed during bazel build", and when I run a bazel query "deps(//sw/device/tests:aes_smoketest)" to see if the lint rules are indeed a dep of a chip-level test target, I do not see that they are. See below screenshot:

So something odd is happening that I need to track down, but until then I did not want to hold up the Bazel transition on something I know how to fix in the iterim. Specifically, I already learned how to cache Python wheels that are downloaded/installed with the pip_install Bazel rule, so I used this same approach here (which works).

I will keep this issue open, and have added a task (see above) to track down this issue to eventually simplify the need for this work around.

[rswarbrick]

Ah, I see. I guess I was asking something even stupider: why are we running the licence checker in Bazel, which I thought was supposed to be for building our software? It's not clear to me that compiling a bunch of C and assembly code requires a linting script that looks at the first few lines of each file...

[timothytrippel]

It is for two reasons: 1) provide developers with a simple bazel run quality-like command that allows them to run all necessary linter scripts/programs on all altered files in the repo locally (before submitting a PR), and 2) to enable CI to eventually (one day) consist of three simple steps (obviously simplifying here):
a) bazel run quality
b) bazel build //...
c) bazel test //...