[CI Deduplication] CI configuration for external repo integration by jayhawk-commits · Pull Request #3188 · ROCm/TheRock

jayhawk-commits · 2026-01-31T07:27:09Z

Add CI configuration for external repository integration

Overview

This PR adds support for configuring CI workflows when external repositories (rocm-libraries, rocm-systems) call TheRock's reusable workflows.

Dependencies

This PR depends on PR #3181, which introduces the external repo detection module (detect_external_repo_config.py).

Architecture

graph TB
    subgraph layer1 [Layer 1 - Foundation]
        pathFilters[configure_ci_path_filters.py<br/>Path filtering only]
        detectConfig[detect_external_repo_config.py<br/>External repo config]
    end
    
    subgraph layer2 [Layer 2 - Orchestration]
        mainCI[configure_ci.py<br/>Main CI - internal and external]
    end
    
    mainCI -->|imports| pathFilters
    mainCI -->|imports| detectConfig

Layer 1: configure_ci_path_filters (path filtering; no orchestration). detect_external_repo_config (repo path, skip patterns, test list).
Layer 2: configure_ci.py – single entry point. Handles both TheRock CI and external-repo CI (EXTERNAL_REPO_NAME, project detection, path-based build decision, cross-product, empty matrix).

Testing

Existing tests in configure_ci_path_filters_test.py and test_configure_ci_external_repos.py; all updated to run against the new structure (external-repo tests import from configure_ci and patch configure_ci.get_git_modified_paths etc.).

Environment variables

EXTERNAL_REPO_NAME (optional): When set, config is built for that external repo (e.g. rocm-libraries).
PROJECTS (optional): Comma-separated project list or "all" (e.g. "projects/rocprim,projects/rocblas").

Introduce detect_external_repo_config.py module as foundational infrastructure for external repository integration. This standalone module will be used by future PRs to support rocm-libraries and rocm-systems integration. Includes unit tests for module functionality.

…tion

…users/jayhawk-commits/pr/ci-config-external

jayhawk-commits · 2026-02-05T20:36:58Z

Need to refactor after some recent changes to configure CI by other developers.

Co-authored-by: Cursor <cursoragent@cursor.com>

HereThereBeDragons · 2026-02-06T16:12:36Z


+  Environment variables (for external repo runs; set by workflows that invoke this for rocm-libraries, etc.):
+  * EXTERNAL_REPO_NAME (optional): Repo name (e.g. "rocm-libraries"). When set, config is built for that external repo.
+  * PROJECTS (optional): Comma-separated project list or "all" (e.g. "projects/rocprim,projects/rocblas").


is this dependent on if it external repo or not? if yes, please add a comment

I have added the requirement detail in the comment.

HereThereBeDragons · 2026-02-06T16:19:42Z

+# --------------------------------------------------------------------------- #
+
+
+def _should_skip_external_build_due_to_paths(


both functions _should_skip_external_build_due_to_paths and should_skip_external_build_due_to_paths are not so long. i would put them in one?

I have combined the two functions. Thanks for the suggestion.

HereThereBeDragons · 2026-02-06T16:25:08Z

+    return not should_build
+
+
+def should_build_external_repo(


lets go away from all the "should" and just name it something like requires_external_repo_build

I have updated the wording to requires.

HereThereBeDragons · 2026-02-06T16:26:00Z

+    paths = get_git_modified_paths(base_ref, cwd=cwd)
+    if paths is not None:
+        paths = list(paths)
+    print("modified_paths (max 200):", paths[:200] if paths else paths)


Fixed the logic here for None paths.

HereThereBeDragons · 2026-02-06T16:27:50Z


-def get_git_modified_paths(base_ref: str) -> Optional[Iterable[str]]:
+def get_git_modified_paths(
+    base_ref: str, cwd: Optional[str] = None


please better naming for cwd like external_repo_path or something like that

I have renamed cwd.

HereThereBeDragons · 2026-02-06T16:49:07Z

+        self.assertEqual(len(result), 1)
+        self.assertEqual(result[0]["family"], "gfx94x")
+        self.assertEqual(result[0]["platform"], "linux")
+        self.assertEqual(result[0]["projects_to_test"], "rocprim,rocblas")


above you expect a list in TestParseProjectsInput - but here we expect a comma separated string?

I've changed the code, so that comma-separated strings are used consistently to match existing CI behaviour of accepting comma-separated strings from workflow_dispatch input.

HereThereBeDragons · 2026-02-06T16:51:55Z

+        self.assertEqual(result[0]["projects_to_test"], "rocprim")
+
+    def test_no_cmake_options_in_result(self):
+        """Test that cmake_options are NOT included in result (full builds only)."""


what is the context here with full builds?

One of the items on our backlog is having rocm-systems PRs do full ROCm build. Right now, rocm-systems PRs skips compilation of libraries. We've seen a few issues now where a rocm-systems change causes issues with a rocm-libraries project. My intention is to increase the scope of builds to full ROCm builds, and keep the test coverage the same as is.

HereThereBeDragons · 2026-02-06T16:53:21Z

+            self.assertIn("build_variant", r)
+            self.assertIn("test_labels", r)
+            self.assertIn("projects_to_test", r)
+            self.assertNotIn("cmake_options", r)


you are not evenl listing any cmake options here in the project_config?

Removed the cmake_options assertion.

HereThereBeDragons · 2026-02-06T16:53:58Z

+        result = cross_product_projects_with_gpu_variants(project_configs, gpu_variants)
+
+        self.assertEqual(len(result), 4)  # 2 projects * 2 variants
+        # Verify structure is preserved


i think you want to check here if the pairing was successful? you could even hardcode the expected results

Hardcoded expected results.

HereThereBeDragons · 2026-02-06T16:58:08Z

    get_all_families_for_trigger_types,
 )
+from detect_external_repo_config import (
+    get_external_repo_path,


i didnt manage to review your other pr. but i would like if some of those names could be a bit more descriptive? e.g get_skip_patterns could be better renamed get_skip_patterns_for_ci

Thank you for the suggestion. I have renamed the functions from the module to be more descriptive.

…ove tests

…ilter helpers

jayhawk-commits · 2026-05-11T22:15:49Z

Multi-arch CI is the path forward. Some of the logic here might be applied, but changes should be recreated as a new PR specific to multi-arch CI.

jayhawk-commits and others added 2 commits January 30, 2026 17:28

Add CI configuration for external repo integration

eed84ed

github-project-automation Bot moved this to TODO in TheRock Triage Jan 31, 2026

github-project-automation Bot added this to TheRock Triage Jan 31, 2026

jayhawk-commits and others added 3 commits January 31, 2026 02:43

Fix trailing whitespace from pre-commit

3f11177

Merge branch 'main' into users/jayhawk-commits/pr/external-repo-detec…

8fa77ac

…tion

Merge branch 'users/jayhawk-commits/pr/external-repo-detection' into …

b15a085

…users/jayhawk-commits/pr/ci-config-external

This was referenced Feb 2, 2026

TheRock CI Deduplication Refactor #2960

Closed

Deduplication and removal of circular calls in CI workflows and scripts between TheRock and super repos #2738

Open

[CI] Move ci path filtering funcs into separate file #3256

Merged

Base automatically changed from users/jayhawk-commits/pr/external-repo-detection to main February 5, 2026 20:35

jayhawk-commits and others added 5 commits February 5, 2026 17:04

Merge origin/main into users/jayhawk-commits/pr/ci-config-external

71db594

Co-authored-by: Cursor <cursoragent@cursor.com>

Merge external-repos into configure_ci; path_filters paths-only

59679c7

Co-authored-by: Cursor <cursoragent@cursor.com>

Simplify external repo CI config and align tests with style guide

10a6c3a

Remove redundant test docstrings

0bf3f39

Remove duplicate function and simplify external repo logic

0e10a11

jayhawk-commits requested a review from HereThereBeDragons February 6, 2026 07:11

HereThereBeDragons reviewed Feb 6, 2026

View reviewed changes

jayhawk-commits and others added 7 commits February 6, 2026 21:08

Standardize projects format to string, rename get_skip_patterns, impr…

c846956

…ove tests

Improve function naming and documentation for external repo CI config

fa33dac

Add external repo CI workflow pattern support and Args docs to path f…

60f2ba1

…ilter helpers

Align paths print with TheRock pattern and clarify empty input test

632008b

Merge origin/main into PR branch to resolve conflicts

0aaf892

Merge branch 'main' into users/jayhawk-commits/pr/ci-config-external

0a7b890

Add support for external repos to define EXTRA_CMAKE_OPTIONS

f72157e

jayhawk-commits closed this May 11, 2026

github-project-automation Bot moved this from TODO to Done in TheRock Triage May 11, 2026

jayhawk-commits deleted the users/jayhawk-commits/pr/ci-config-external branch May 11, 2026 22:18

		# --------------------------------------------------------------------------- #


		def _should_skip_external_build_due_to_paths(

Conversation

jayhawk-commits commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add CI configuration for external repository integration

Overview

Dependencies

Architecture

Testing

Environment variables

Uh oh!

jayhawk-commits commented Feb 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jayhawk-commits commented May 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jayhawk-commits commented Jan 31, 2026 •

edited

Loading