Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8e8dac8
build(bazel): add cloud-tasks maven closure and grpc-java to the root…
balajinvda Jul 22, 2026
6ab9b12
build(bazel): regenerate nv-boot NOTICE after the cloud-tasks re-pin
balajinvda Jul 22, 2026
8e5f501
feat(cloud-tasks): add the Java cloud-tasks service under Bazel
balajinvda Jul 22, 2026
04e75eb
chore(cloud-tasks): drop internal Maven server ref from the public pom
balajinvda Jul 22, 2026
b530721
fix(cloud-tasks): exclude docker integration tests from CI + index NO…
balajinvda Jul 22, 2026
0a9c9bc
ci(bazel): run integration tests in a DinD lane instead of skipping them
balajinvda Jul 22, 2026
db5ec4d
ci(bazel): run the integration lane on the bare runner, not container…
balajinvda Jul 22, 2026
39ad99a
fix(cloud-tasks): materialize compose assets so the Cassandra schema …
balajinvda Jul 22, 2026
03ee728
fix(cloud-tasks): address the two CodeQL alerts
balajinvda Jul 22, 2026
4368ee2
fix(cloud-tasks): sync OpenAPI schema annotations from upstream 4c6f57c9
balajinvda Jul 23, 2026
5994724
test(cloud-tasks): normalize extraction base so the zip-slip guard is…
balajinvda Jul 23, 2026
7436dec
feat(java): standalone nvcf_java_rules + nv_boot_parent Bazel modules
balajinvda Jul 23, 2026
c2c656e
Merge branch 'main' of https://github.com/NVIDIA/nvcf into consolidate
balajinvda Jul 23, 2026
a1bcbb5
fix(java): use the image-local JDK 25 instead of a hermetic remotejdk…
balajinvda Jul 23, 2026
257a536
fix(java): align cloud-tasks + nv-boot-parent to the canonical single…
balajinvda Jul 23, 2026
1d83ba9
Merge commit '006718bb840cd90d4a9f55b7f95199393669bab1' into verify343
balajinvda Jul 23, 2026
5c7cfc8
refactor(java): fold nv-boot-parent + cloud-tasks into the single roo…
balajinvda Jul 23, 2026
60b5516
ci(bazel): add root-scoped lane mode; give Java its own matrix rows
balajinvda Jul 23, 2026
5ceb1e4
ci(bazel): give the integration lane a JDK 25 for local_jdk
balajinvda Jul 23, 2026
5ec0a38
fix(ci): resolve cloud-tasks test failures in the GHA bazel lanes
balajinvda Jul 23, 2026
b449d03
ci(bazel): validate nv-boot-parent consumers on framework changes
balajinvda Jul 23, 2026
32b34c6
ci(bazel): run the full test matrix on every PR (no change-aware skip…
balajinvda Jul 23, 2026
934158a
ci(bazel): per-service Docker lanes so no subtree lane runs zero tests
balajinvda Jul 23, 2026
dacf015
ci(bazel): rename per-service Docker lanes to bazel (<id>)
balajinvda Jul 23, 2026
833220b
build(bazel): enforce root Java module ownership
sanjay-saxena Jul 23, 2026
36911cd
Complete Cloud Tasks Bazel parity validation
sanjay-saxena Jul 23, 2026
8b33aec
feat(bazel): formalize layered Java NOTICE generation
sanjay-saxena Jul 24, 2026
c8caf29
Automate Java Bazel CI registration and artifacts
sanjay-saxena Jul 24, 2026
3a0dcba
docs(bazel): complete monorepo Java guidance
sanjay-saxena Jul 24, 2026
6934b8c
build: make Zig toolchain root-only
sanjay-saxena Jul 24, 2026
bfb3976
Preserve nvct-core fixture resources for external consumers
sanjay-saxena Jul 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions .bazel_downloader_config
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Prefer the Google Maven Central mirror for repository-rule downloads that
# hardcode repo1.maven.org. Coursier dependencies still use MODULE.bazel's
# maven.install repositories.
rewrite (repo1.maven.org)/(maven2/.*) https://maven-central.storage-download.googleapis.com/$2
18 changes: 16 additions & 2 deletions .bazelignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,12 @@
# with our bzlmod setup. Re-enable per-subtree as Phase B brings each subtree
# native.
src/compute-plane-services
src/control-plane-services
# src/control-plane-services is carved out per-subtree: cloud-tasks is native
# (Phase B) and must stay loadable, while these siblings still carry nested
# MODULE.bazel / WORKSPACE-style BUILD files that conflict with bzlmod.
src/control-plane-services/function-autoscaler
src/control-plane-services/helm-reval
src/control-plane-services/nats-auth-callout
src/invocation-plane-services
src/libraries/python
# stargate is a nested Bazel module (own MODULE.bazel under
Expand All @@ -45,7 +50,16 @@ docs
fern
infra
migrations
tools

# Note: tools/ is not ignored here so //tools/bazel/java (the root-owned Java
# build helpers) is loadable. Gazelle is still kept out of tools/ via the
# gazelle:exclude directive in the root BUILD.bazel.

# The Java subtrees (src/libraries/java/nv-boot-parent and
# src/control-plane-services/cloud-tasks) are part of the single root module:
# plain source dirs with BUILD.bazel only, no nested MODULE.bazel. They are
# intentionally loadable so root labels traverse them. Gazelle is kept out via
# the root BUILD.bazel gazelle:exclude directives.

# Vendored dependencies inside the two Phase 1 subtrees. Bazel resolves these
# from MODULE.bazel via Gazelle's go_deps extension, not from on-disk vendor.
Expand Down
25 changes: 20 additions & 5 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -68,15 +68,30 @@ common --incompatible_enable_proto_toolchain_resolution
build --workspace_status_command=./tools/workspace_status.sh
build --nostamp

# Java: build and run against a hermetic remotejdk 21 regardless of the host
# JDK, so Java service builds are reproducible on any developer machine and in
# CI. Language version 21 is the current Spring Boot 3.x baseline.
# Java baseline: Java 25 for the whole monorepo (nv-boot-parent and the Java
# control-plane services). local_jdk is the approved runtime. The pinned
# bazel-ci image supplies Temurin 25 through JAVA_HOME; the bare
# Docker-integration lane supplies it through actions/setup-java. The Java
# build and test jobs prove that both environments satisfy this contract.
# Full javac (header compilation off) is required because Lombok-generated APIs
# in the imported projects are not compatible with Bazel's Turbine header
# compiler.
build --java_language_version=25
build --java_runtime_version=remotejdk_25
build --tool_java_language_version=25
build --tool_java_runtime_version=remotejdk_25
build --java_runtime_version=local_jdk
build --tool_java_runtime_version=local_jdk
build --java_header_compilation=false

# rules_spring 2.6.3 (cloud-tasks Spring Boot packaging) still uses bare Java
# rules/providers in its own BUILD and .bzl files; Bazel 9 requires these
# symbols to be autoloaded for that external repo.
common --incompatible_autoload_externally=+@rules_java

# Downloader rewrites for repository-rule downloads (e.g. rules that hardcode
# repo1.maven.org). Coursier maven.install still uses the repositories declared
# in MODULE.bazel. Public, credentials-free mappings only.
common --downloader_config=.bazel_downloader_config

test --test_output=errors
# Bazel's sandbox strips HOME, but a lot of Go code (cobra/viper config,
# user state files, git cache lookups) blows up without it. Point HOME at
Expand Down
294 changes: 265 additions & 29 deletions .github/workflows/bazel.yml

Large diffs are not rendered by default.

216 changes: 193 additions & 23 deletions BAZEL.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
# Bazel in the NVCF monorepo

This file is the contributor-facing guide for the Bazel build path in the
NVCF umbrella repo. Bazel is the build engine for the native subtrees in
Phase 1; upstream-owned subtrees keep their existing build paths until they go
native.
NVCF umbrella repository. Bazel is the build engine for onboarded subtrees;
upstream-owned subtrees keep their existing build paths until they are
explicitly integrated.

## Phase 1 scope
## Current Scope

Bazel currently builds, tests, and packages:

- `src/clis/nvcf-cli` (Go binary + multi-platform release matrix + OCI image)
- `src/libraries/go/lib` (Go library, 92 targets)
- `src/libraries/java/nv-boot-parent` (Java framework libraries and tests)
- `src/control-plane-services/cloud-tasks` (Java libraries, tests, and Spring
Boot application)

Subtrees listed in `imports.yaml` with `authoritative_source: upstream` are
intentionally excluded via
`.bazelignore` and `# gazelle:exclude` directives in the root `BUILD.bazel`.
They will be onboarded one at a time as Phase B in separate MRs.
Other upstream-owned subtrees remain excluded until they are onboarded one at
a time. `nv-boot-parent` and onboarded Java service directories are folded
into the root Bazel module and are not nested Bazel workspaces.

## One-time setup

Expand Down Expand Up @@ -62,10 +64,155 @@ the cross-toolchain on first use.

### Common environment

The repo expects Bazel 8.6.0 (pinned in `.bazelversion`). Bazelisk handles
The repo expects Bazel 9.1.1 (pinned in `.bazelversion`). Bazelisk handles
the download automatically; do not install Bazel via apt or brew directly,
as that pins a different version.

Java targets use Java 25 with the root `.bazelrc` setting
`--java_runtime_version=local_jdk`. The pinned containerized CI image supplies
Temurin 25 through `JAVA_HOME`. The Docker-host lane downloads and configures
Temurin 25 through the workflow's `actions/setup-java@v4` step.

For local Java work, install an organization-approved full JDK 25 and point
`JAVA_HOME` at it:

```bash
export JAVA_HOME="<path-to-jdk-25>"
export PATH="${JAVA_HOME}/bin:${PATH}"
java --version
```

Java services with Testcontainers tests and the nv-boot Cassandra tests also
require a running Docker daemon. Use Docker Desktop on macOS or Docker Engine
on Linux.

`bazel info java-home` reports the Java runtime used to run the Bazel server.
That directory is not guaranteed to contain command-line tools such as
`javac`, so it is not the right way to inspect the Java toolchains selected for
build actions. Query those toolchains directly:

```bash
bazel cquery @bazel_tools//tools/jdk:current_java_runtime \
--output=starlark \
--starlark:expr='str(providers(target)["ToolchainInfo"].java_runtime.version)'

bazel cquery @bazel_tools//tools/jdk:current_host_java_runtime \
--output=starlark \
--starlark:expr='str(providers(target)["ToolchainInfo"].java_runtime.version)'
```

Both commands must print `25`. The normal Java build then proves that the
compiler toolchain works; a separate `javac` command under
`bazel info java-home` is neither required nor expected.

## Java In The Root Module

`nv-boot-parent` and onboarded Java services are ordinary source directories
inside the single root `nvcf` Bazel module. They do not have nested
`MODULE.bazel`, `.bazelrc`, `.bazelversion`, downloader configuration, or
dependency lockfiles. Run their Bazel commands from this repository root.

For someone familiar with Maven, these root files divide responsibilities that
often live in a parent POM, Maven settings, and the local repository:

| Root file | Purpose |
|---|---|
| `.bazelversion` | Tells Bazelisk which Bazel release to run. |
| `.bazelrc` | Supplies shared Bazel flags, Java 25 toolchain settings, and downloader configuration. |
| `.bazel_downloader_config` | Redirects supported external downloads through approved mirrors. It does not declare dependencies. |
| `MODULE.bazel` | Declares Bazel rule modules, BOM imports, and the roots of the shared third-party Java graph. |
| `maven_install.json` | Locks resolved third-party Java artifacts, relationships, repositories, and checksums. Its name describes Maven-compatible coordinates; it does not run Maven. |
| `MODULE.bazel.lock` | Locks Bzlmod modules and module-extension evaluation. It is separate from the Java artifact lock. |

Commit changes to these files together when one dependency update affects more
than one of them. Do not edit either lockfile manually.

All Java components use the root-owned `@nv_third_party_deps` hub for external
jars. The hub contains third-party artifacts only. `nv-boot-parent` and each
service remain first-party source targets referenced with direct labels such
as:

```text
//src/libraries/java/nv-boot-parent/nv-boot-starter-core:nv_boot_starter_core
//src/control-plane-services/<service-directory>/<module>:<target>
```

Set a portable output root once per local shell:

```bash
export BAZEL_OUTPUT_USER_ROOT="${TMPDIR:-/tmp}/nvcf-bazel-cache"
export JAVA_SERVICE_DIR="src/control-plane-services/<service-directory>"
export JAVA_APP_MODULE="<spring-boot-app-module>"
```

Replace the angle-bracket placeholders with the service directory and its
Spring Boot application module. The component's own `BAZEL.md` supplies its
exact values and targets.

Build or test the complete framework or one Java service:

```bash
bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
build //src/libraries/java/nv-boot-parent/...

bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
test //src/libraries/java/nv-boot-parent/... \
--cache_test_results=no \
--test_output=errors

bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
build "//${JAVA_SERVICE_DIR}/..."

bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
test "//${JAVA_SERVICE_DIR}/..." \
--cache_test_results=no \
--test_output=errors
```

When the selected scope includes Testcontainers tests, the complete suite
requires a running Docker daemon. `bazel test` automatically builds the code
needed by the selected tests; a separate build is useful for compile-only
feedback and for non-test products such as the Spring Boot app jar.

Build the selected service's executable app jar with:

```bash
bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
build "//${JAVA_SERVICE_DIR}/${JAVA_APP_MODULE}:app"
```

Its output is:

```text
bazel-bin/src/control-plane-services/<service-directory>/<spring-boot-app-module>/app.jar
```

Real Java test, JUnit, and JaCoCo outputs are under each target's
`bazel-testlogs/<component>/<module>/tests/test.outputs` directory. The
component guides provide commands for one module, class, or method and for
NOTICE, OSRB, Docker, and Maven coexistence:

```text
src/libraries/java/nv-boot-parent/BAZEL.md
src/control-plane-services/<service-directory>/BAZEL.md
```

When the shared third-party Java graph changes, repin it from the root:

```bash
REPIN=1 bazel --output_user_root="${BAZEL_OUTPUT_USER_ROOT}" \
run @nv_third_party_deps//:pin
```

Java CI registration is component-local. Each component owns one
`bazel-java-ci.json`; `.github/workflows/bazel.yml` discovers those files and
derives shared-Java triggers, framework-to-service validation, CI execution
environment, and artifact upload. Use `ci_lane: docker-host` when any component
test requires Docker and `ci_lane: build-container` otherwise. Java root scope
is implicit and is not a descriptor option. The detector supports
dependency-aware selection, but current policy deliberately runs the full
matrix on every PR and push for regression safety.

## Day-to-day commands

### Build
Expand Down Expand Up @@ -135,7 +282,7 @@ bazel run //:gazelle
bazel mod tidy
```

Gazelle is configured to skip everything outside Phase 1 scope, so it will
Gazelle is configured to skip everything outside the current native scope, so it will
not touch upstream-owned subtrees or vendored directories.

#### Rust equivalent
Expand Down Expand Up @@ -282,12 +429,31 @@ build).

## CI

Two Bazel-aware jobs in the root `.gitlab-ci.yml`:
The public GitHub Bazel matrix in `.github/workflows/bazel.yml` consumes
`ghcr.io/nvidia/nvcf/bazel-ci:0.12.0`. That image is built in the internal
[`nvcf/bazel-ci-templates`](https://gitlab-master.nvidia.com/nvcf/bazel-ci-templates)
project, stamped with a version, and mirrored to GHCR. The mirror is currently
manual; automation is planned. To change the image's Bazel, Java, or operating
system tooling, update the internal template first, publish and mirror a new
tag, and only then update the pinned `container.image` here.

The root `ci/Dockerfile.bazel` and
`.github/workflows/bazel-ci-image.yml` are legacy files and are not the
authoritative producer for the image used by the matrix. Do not use them to
reason about the contents of `bazel-ci:0.12.0`.

The detect job also enforces the single-module import boundary:

```bash
bash tools/ci/check-java-import-boundaries
```

Run this after refreshing `nv-boot-parent` or an onboarded Java service. It
fails when a standalone Bazel root file, lockfile, workspace file, or migration
directory is reintroduced under an imported subtree.

The existing internal Bazel jobs include:

- `bazel-ci-image`: rebuilds `ci/Dockerfile.bazel` via buildah and pushes
to `${CI_REGISTRY_IMAGE}/bazel-ci:<ref-slug>`. Triggers only when
`ci/Dockerfile.bazel` or `.bazelversion` changes (or when a web pipeline
is run with `$REBUILD_BAZEL_IMAGE` set).
- `bazel-smoke`: pulls the image, runs `bazel info release`, then
`bazel build --config=remote //src/libraries/go/lib/...
//src/clis/nvcf-cli:image_index` and `bazel mod graph`. It does not
Expand Down Expand Up @@ -328,7 +494,7 @@ build).

## Adding a new Go module

For native subtrees outside Phase 1 scope today:
For native subtrees outside the current scope:

1. Add the module path to `go.work.bazel` under `use (...)`.
2. Add or update its `go.mod`.
Expand All @@ -339,9 +505,13 @@ For native subtrees outside Phase 1 scope today:
5. Run `bazel run //:gazelle` then `bazel mod tidy`.
6. `bazel build //path/to/subtree/...` to validate.

For upstream-owned subtrees (`authoritative_source: upstream` in
`imports.yaml`), Bazel files belong upstream so they survive the next commit-pin
refresh.
The public checkout does not contain the internal source-mirroring
configuration. For an upstream-owned subtree, distinguish between source files
that continue to mirror from the standalone repository and monorepo-native
Bazel overlays. The import process must exclude standalone `MODULE.bazel`,
lockfiles, `.bazelrc`, `.bazelversion`, downloader config, dependency hub,
and `bazel-enablement` content. Root-module BUILD adaptations and monorepo
agent/documentation overlays must be preserved during refreshes.

## Per-service publish cadence

Expand Down Expand Up @@ -414,23 +584,23 @@ maintainers; centralising it would couple unrelated release decisions.
- `bazel info release` blocks for >30 s on first run: it is downloading the
pinned Bazel binary. One-time cost.

## Phase B status
## Additional Subtree Rollout

Per-service rollout state for upstream-owned subtrees is tracked in an internal
plan that references upstream GitLab URLs and per-service rollout state that
does not belong in the public mirror, including which upstream MRs are open,
which are merged, and which umbrella `imports.yaml` bumps have landed. Update
that internal plan as each service moves through the playbook.

## Out of scope (Phase B and later)
## Out of scope (Later Phase)

- Wiring upstream-owned subtrees listed in `imports.yaml`. One MR per upstream
owner. See the tracker.
- Migrating goreleaser-driven release stages
(archive/package/publish/ngc-push) onto Bazel-native equivalents (e.g.
`pkg_tar`, `oci_push`, custom rules for NGC). Today the artifact
contracts are preserved via copy-from-bazel-bin shims in CI.
- Coverage report generation in CI. `bazel coverage` works locally; CI
parsing of coverage output is deferred.
- Go coverage report publication in CI. Java JUnit and JaCoCo reports are
already generated and uploaded by their component lanes.
- Lint integration. `golangci-lint` still runs as a separate job and is
not yet wrapped into a Bazel rule.
Loading
Loading