Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for osx-arm64 builds #46775

Merged
merged 53 commits into from
Jul 10, 2024
Merged

feat: support for osx-arm64 builds #46775

merged 53 commits into from
Jul 10, 2024

Conversation

aliciaaevans
Copy link
Contributor

@aliciaaevans aliciaaevans commented Mar 26, 2024

To Do:

  • Bioconda-Utils artifact fetching feat: fetch artifacts from GitHub Actions bioconda-utils#973
  • GitHub Actions and CircleCI workflow to upload to channel on merge of PR.
  • Include osx-arm64 in Bulk.yml GitHub Actions workflow
  • BiocondaBot comment artifact fetching feat: download github artifacts and condense comment bioconda-containers#79
  • Quick exit if the additional platform is not specified, similar to CircleCI workflow for linux-aarch64. Moved script to scripts directory and updated to be generic for any CI platform.
  • Update documentation OSX-Arm64 updates bioconda-docs#24
  • Compare CircleCI and GitHub Actions performance. Findings:
    • CircleCI has better performance on a single osx-arm64 build, so that will be used for PRs and merges to master, along with linux-aarch64.
    • GitHub Actions has a much longer timeout, so that will be used for osx-arm64 bulk builds along with the bulk linux-64 and osx-64 builds.
    • The PR and master pipelines for GitHub Actions are manually disabled, but I added the new jobs there in case we ever need to reenable them.

.github/workflows/PR-arm.yml Outdated Show resolved Hide resolved
.github/workflows/PR-arm.yml Outdated Show resolved Hide resolved
@aliciaaevans
Copy link
Contributor Author

bioconda/bioconda-utils#967 will avoid needing to set the SDK version in the workflow.

@aliciaaevans
Copy link
Contributor Author

bioconda/bioconda-utils#967 will avoid needing to set the SDK version in the workflow.

Actually this did not fully work becauseMACOSX_DEPLOYMENT_TARGET needs to be set before run_conda_forge_build_setup is run.

@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bioawk-1.0-he4a0461_11.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h45fc8d7_11.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bioawk-1.0-he4a7b4a_11.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/b6aaff9f-75d6-47e7-8187-4f6efd1d4b91/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bioawk 1.0--he4a0461_11 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_11.tar.gz | docker load

@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
linux-64 bamtools-2.5.2-hdcf5f25_4.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
linux-64 bioawk-1.0-he4a0461_11.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bamtools-2.5.2-h447d7a5_4.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h10309d6_11.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/ee37c488-b191-4491-9477-e893ac067ca7/artifacts/0/tmp/artifacts/packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/ee37c488-b191-4491-9477-e893ac067ca7/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bamtools-2.5.2-h7f4e536_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/46f12a5c-5fe0-4c78-875b-53e09a0e5303/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/46f12a5c-5fe0-4c78-875b-53e09a0e5303/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bamtools 2.5.2--hdcf5f25_4 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bamtools:2.5.2--hdcf5f25_4.tar.gz | docker load
bioawk 1.0--he4a0461_11 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_11.tar.gz | docker load

@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bamtools-2.5.2-hdcf5f25_4.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
linux-64 bioawk-1.0-he4a0461_11.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bamtools-2.5.2-h447d7a5_4.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h10309d6_11.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/82ea5dfb-971b-42bb-add2-c24f1aa8d169/artifacts/0/tmp/artifacts/packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/82ea5dfb-971b-42bb-add2-c24f1aa8d169/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bamtools-2.5.2-h7f4e536_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/d226436a-3716-457b-b4c7-0ad3d8bbf8c3/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/d226436a-3716-457b-b4c7-0ad3d8bbf8c3/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bamtools 2.5.2--hdcf5f25_4 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bamtools:2.5.2--hdcf5f25_4.tar.gz | docker load
bioawk 1.0--he4a0461_11 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_11.tar.gz | docker load

@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bamtools-2.5.2-hdcf5f25_4.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
linux-64 bioawk-1.0-he4a0461_11.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bamtools-2.5.2-h447d7a5_4.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h10309d6_11.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 osx-arm64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/14d896e0-28aa-4bc7-b170-e6ac1a0db6e7/artifacts/0/tmp/artifacts/packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/14d896e0-28aa-4bc7-b170-e6ac1a0db6e7/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bamtools-2.5.2-h7f4e536_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/e8962f6a-9b8e-42de-b26a-69ad790fbb12/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/e8962f6a-9b8e-42de-b26a-69ad790fbb12/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bamtools 2.5.2--hdcf5f25_4 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bamtools:2.5.2--hdcf5f25_4.tar.gz | docker load
bioawk 1.0--he4a0461_11 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_11.tar.gz | docker load

@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bamtools-2.5.2-hdcf5f25_4.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
linux-64 bioawk-1.0-he4a0461_11.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bamtools-2.5.2-h447d7a5_4.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h10309d6_11.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/d1f1dd79-c173-4fc6-802d-9b68fae7c771/artifacts/0/tmp/artifacts/packages <package name>
osx-arm64 bioawk-1.0-hcb76912_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/d1f1dd79-c173-4fc6-802d-9b68fae7c771/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bamtools-2.5.2-h7f4e536_4.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/cc255e8f-fe4c-4642-a6f2-626c3eca8277/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_11.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/cc255e8f-fe4c-4642-a6f2-626c3eca8277/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bamtools 2.5.2--hdcf5f25_4 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bamtools:2.5.2--hdcf5f25_4.tar.gz | docker load
bioawk 1.0--he4a0461_11 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_11.tar.gz | docker load

@aliciaaevans aliciaaevans changed the title (WIP) feat: support for osx-arm64 builds feat: support for osx-arm64 builds Jul 9, 2024
@aliciaaevans aliciaaevans added MacOS related osx-arm64 Related to adding osx-arm64 support and removed WIP labels Jul 9, 2024
aliciaaevans added a commit to bioconda/bioconda-common that referenced this pull request Jul 9, 2024
.circleci/config.yml Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
.github/workflows/Bulk.yml Outdated Show resolved Hide resolved
.github/workflows/PR.yml Show resolved Hide resolved
.github/workflows/PR.yml Outdated Show resolved Hide resolved
.github/workflows/master.yml Outdated Show resolved Hide resolved
aliciaaevans and others added 4 commits July 10, 2024 10:51
This does add a little clutter from conda activate, but it could be useful for debugging some of the osx-specific stuff.

Co-authored-by: Martin Grigorov <[email protected]>
@aliciaaevans aliciaaevans marked this pull request as ready for review July 10, 2024 15:57
@aliciaaevans
Copy link
Contributor Author

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 bamtools-2.5.2-hdcf5f25_5.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
linux-64 bioawk-1.0-he4a0461_12.tar.bz2 LinuxArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the LinuxArtifacts directory: conda install -c ./packages <package name>
osx-64 bamtools-2.5.2-h447d7a5_5.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-64 bioawk-1.0-h10309d6_12.tar.bz2 OSXArtifacts.zip Azure
showYou may also use conda to install after downloading and extracting the zip file. From the OSXArtifacts directory: conda install -c ./packages <package name>
osx-arm64 bamtools-2.5.2-h6057758_5.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/701a80ab-e885-4871-8c1c-e0b3791449f8/artifacts/0/tmp/artifacts/packages <package name>
osx-arm64 bioawk-1.0-hcb76912_12.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/701a80ab-e885-4871-8c1c-e0b3791449f8/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bamtools-2.5.2-h7f4e536_5.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/894a254e-d9f0-4d55-8b9c-f52203cd94f9/artifacts/0/tmp/artifacts/packages <package name>
linux-aarch64 bioawk-1.0-h73052cd_12.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/894a254e-d9f0-4d55-8b9c-f52203cd94f9/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
bamtools 2.5.2--hdcf5f25_5 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bamtools:2.5.2--hdcf5f25_5.tar.gz | docker load
bioawk 1.0--he4a0461_12 Azure
showImages for Azure are in the LinuxArtifacts zip file above.gzip -dc LinuxArtifacts/images/bioawk:1.0--he4a0461_12.tar.gz | docker load

@aliciaaevans aliciaaevans merged commit 21fbd60 into master Jul 10, 2024
6 checks passed
@aliciaaevans aliciaaevans deleted the osx-arm-builds branch July 10, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MacOS related osx-arm64 Related to adding osx-arm64 support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants