Skip to content

Add support for Cirun on self-hosted GHA runners#1703

Merged
isuruf merged 75 commits into
conda-forge:mainfrom
jaimergp:gha-self-hosted-cirun
Nov 6, 2023
Merged

Add support for Cirun on self-hosted GHA runners#1703
isuruf merged 75 commits into
conda-forge:mainfrom
jaimergp:gha-self-hosted-cirun

Conversation

@jaimergp

@jaimergp jaimergp commented Dec 19, 2022

Copy link
Copy Markdown
Member

@aktech aktech force-pushed the gha-self-hosted-cirun branch from bf49fcc to 7b0c3e3 Compare August 22, 2023 11:33
@isuruf

isuruf commented Oct 23, 2023

Copy link
Copy Markdown
Member

It's a bug from #1752

Comment thread conda_smithy/cirun_utils.py Outdated
Comment thread conda_smithy/cirun_utils.py
Comment thread conda_smithy/cirun_utils.py Outdated
Comment thread conda_smithy/cirun_utils.py Outdated
@isuruf isuruf marked this pull request as ready for review October 31, 2023 13:58
@isuruf isuruf requested a review from a team as a code owner October 31, 2023 13:58
@beckermr

Copy link
Copy Markdown
Member

I don't think I have expertise to review this PR. However, can we make the docs PR sooner rather than later? I think that will help others figure out what to do with all of these new features.

@isuruf isuruf changed the title (WIP) Add support for Cirun on self-hosted GHA runners Add support for Cirun on self-hosted GHA runners Oct 31, 2023
@isuruf

isuruf commented Oct 31, 2023

Copy link
Copy Markdown
Member

Docs PR: conda-forge/conda-forge.github.io#2025

@isuruf

isuruf commented Nov 2, 2023

Copy link
Copy Markdown
Member

@conda-forge/core, ready for a review

@ocefpaf

ocefpaf commented Nov 6, 2023

Copy link
Copy Markdown
Member

I don't think I have expertise to review this PR.

Same here but we can take a pragmatic approach by answering the following questions:

  • Does this break existing code? Doesn't seem like it, so +1 to merge.
  • Does this leak anything that shouldn't? Also doesn't seem like it, another OK to merge.

If both my answers above make sense then we can ask: Does it work for those testing it? Or do you need this merge to put it in production? If so, let's merge.

However, can we make the docs PR sooner rather than later? I think that will help others figure out what to do with all of these new features.

+100

@jaimergp

jaimergp commented Nov 6, 2023

Copy link
Copy Markdown
Member Author

Basic docs have been merged. There's a chance we need to write more, but for now I think they are sufficient to get us unblocked here and merge to try things.

Comment thread conda_smithy/cirun_utils.py
Comment thread conda_smithy/configure_feedstock.py Outdated
if config["test"] is None:
config["test"] = "all"

if config["github_actions"]["cancel_in_progress"] is None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning of this? Why bind cancel_in_progress to whether the runners are self-hosted?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To save CI resources by default. Self-hosted runners are usually going to be scarce or costly. If a user wants to not cancel-in-progress, they can explicitly say so, but by default it will cancel previous jobs when self-hosted runners are in use.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also cancel hosted ones, too. They are scarce (only a limited pool of them) and can also be costly if you are not using your free minutes (or a runner that falls into the free minutes).

I would use the rather simple option of

if config["github_actions"]["cancel_in_progress"] is None:
    config["github_actions"]["cancel_in_progress"] = True

Probably you can just set the default to False?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the cancel in progress here is configured to cancel all jobs in the same build. For eg: Linux builds will get cancelled if windows builds fail.

@isuruf isuruf merged commit b1c5fca into conda-forge:main Nov 6, 2023
channel_target.startswith("conda-forge ")
and provider_name == "github_actions"
and not forge_config["github_actions"]["self_hosted"]
and os.path.basename(forge_dir) not in SERVICE_FEEDSTOCKS

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaimergp why was this line removed?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want it to be self-hosted or in service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants