Skip to content

Adding monorepo configurations to TheRock#695

Merged
geomin12 merged 20 commits into
mainfrom
users/geomin12/monorepo-therock-test
May 23, 2025
Merged

Adding monorepo configurations to TheRock#695
geomin12 merged 20 commits into
mainfrom
users/geomin12/monorepo-therock-test

Conversation

@geomin12
Copy link
Copy Markdown
Contributor

@geomin12 geomin12 commented May 22, 2025

Changes:

  • For external repos, we upload to therock-artifacts-external bucket
  • Enabling extra cmake options for linux builds
  • Adding a script to determine what changes were made in monorepo and determine which flags/tests to run

Progress on #665

@geomin12 geomin12 changed the title Users/geomin12/monorepo therock test Adding monorepo configurations to TheRock May 22, 2025
- name: Upload Artifacts
run: |
aws s3 cp build/artifacts/ s3://therock-artifacts/${{github.run_id}}-linux/ \
aws s3 cp build/artifacts/ s3://${{env.BUCKET}}/${{github.run_id}}-linux/ \
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently, for external repos (ex: rocm-libraries), this still uploads all to the dir /.

After this PR, I plan to move this upload items into script in another PR and adding env.REPO to there to allow therock-external/rocm-libraries/123456-linux

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation - def a good thing we need to fix.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would therock-external correspond to env.BUCKET and rocm-libraries to env.REPO then? From a naming perspective, if env.REPO just specifies a subdir in the bucket you could call it env.S3_SUBDIR to make the purpose more clear.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct! Although repo is not used (yet) so I will add this in the next PR!

@geomin12 geomin12 requested review from amd-chrissosa and marbre May 22, 2025 18:33
Comment thread .github/actions/setup_test_environment/action.yml Outdated
@amd-chrissosa
Copy link
Copy Markdown
Contributor

Biggest concern is adding another bucket if we can avoid it - getting too many buckets as is.

Comment thread build_tools/fetch_artifacts.py Outdated
PLATFORM = platform.system().lower()

BUCKET = os.getenv("BUCKET", "therock-artifacts")
REPO = os.getenv("REPO", "")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REPO seems unused in this script.

- name: Upload Artifacts
run: |
aws s3 cp build/artifacts/ s3://therock-artifacts/${{github.run_id}}-linux/ \
aws s3 cp build/artifacts/ s3://${{env.BUCKET}}/${{github.run_id}}-linux/ \
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would therock-external correspond to env.BUCKET and rocm-libraries to env.REPO then? From a naming perspective, if env.REPO just specifies a subdir in the bucket you could call it env.S3_SUBDIR to make the purpose more clear.

Comment thread .github/workflows/build_linux_packages.yml Outdated
repository: "ROCm/TheRock"

- name: "Checking out repository for rocm-libraries"
if: ${{ github.repository == 'ROCm/rocm-libraries' }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With view on more monorepos we might want to make this more flexible but also need take care of PRs from forks (either to one of the monorepos or TheRock). Maybe add a TODO note for now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could actually make it github.event.repository.name == TheRock in order to work with both forks + normal so I will do that!

Copy link
Copy Markdown
Member

@marbre marbre May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy if that works for all the potential use cases. I think we need to cover PRs

  • from within TheRock
  • from forks of TheRock targeting TheRock
  • form within a monorepo (tested via TheRock)
  • form forks of a monorepo targeting the monorepo (tested via TheRock)

I am not familiar enough with GH's event API thus not saying that what you plan doesn't work :)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should work I expect.

@geomin12 geomin12 requested review from amd-chrissosa and marbre May 22, 2025 21:32
@amd-chrissosa
Copy link
Copy Markdown
Contributor

I'm good with this once you are ready to remove the geo parts. Just poke me for an approval

@geomin12 geomin12 marked this pull request as ready for review May 23, 2025 02:46
@geomin12 geomin12 merged commit a742718 into main May 23, 2025
34 of 39 checks passed
@geomin12 geomin12 deleted the users/geomin12/monorepo-therock-test branch May 23, 2025 05:23
@github-project-automation github-project-automation Bot moved this from TODO to Done in TheRock Triage May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants