ci: add a github action ci to test Rust std::autodiff #2430

sgasho · 2025-08-31T05:46:44Z

closes: #2429
related: rust-lang/rust#145899

result of act dry-run

act -n -W .github/workflows/enzyme-rust.yml -P self-hosted=ubuntu-latest -P Linux=ubuntu-latest -P X64=ubuntu-latest -P 32-core=ubuntu-latest
INFO[0000] Using docker host 'unix:///var/run/docker.sock', and daemon socket 'unix:///var/run/docker.sock' 
WARN  ⚠ You are using Apple M-series chip and you have not specified container architecture, you might encounter issues while running act. If so, try running it with '--container-architecture linux/amd64'. ⚠  
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Set up job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] 🚀  Start image=ubuntu-latest
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Set up job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] 🚀  Start image=ubuntu-latest
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   🐳  docker pull image=ubuntu-latest platform= username= forcePull=true
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   🐳  docker pull image=ubuntu-latest platform= username= forcePull=true
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   🐳  docker create image=ubuntu-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   🐳  docker run image=ubuntu-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Set up job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   🐳  docker create image=ubuntu-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   🐳  docker run image=ubuntu-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Set up job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] 🧪  Matrix: map[llvm:19]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] 🧪  Matrix: map[llvm:20]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Install LLVM and dependencies
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Install LLVM and dependencies
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Install LLVM and dependencies [3.041958ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Install LLVM and dependencies [3.560292ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Install Rust toolchain
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Install Rust toolchain
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Install Rust toolchain [2.436583ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Install Rust toolchain [2.396541ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Checkout Enzyme
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Checkout Enzyme
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Checkout Enzyme [1.037666ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Checkout Enzyme [1.382292ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Build Enzyme
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Build Enzyme
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Build Enzyme [2.28875ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Build Enzyme [2.499958ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Clone and configure Rust compiler
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Clone and configure Rust compiler
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Clone and configure Rust compiler [2.189917ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Clone and configure Rust compiler [2.093583ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Build Rust compiler stage 1
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Build Rust compiler stage 1
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Build Rust compiler stage 1 [2.484125ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Build Rust compiler stage 1 [2.589625ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Main Run Rust autodiff tests
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Main Run Rust autodiff tests
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Main Run Rust autodiff tests [2.293625ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] ⭐ Run Complete job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] Cleaning up container for job Rust Autodiff Tests LLVM 20
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Main Run Rust autodiff tests [2.496792ms]
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] ⭐ Run Complete job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] Cleaning up container for job Rust Autodiff Tests LLVM 19
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1]   ✅  Success - Complete job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 19-1] 🏁  Job succeeded
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2]   ✅  Success - Complete job
*DRYRUN* [Enzyme Rust CI/Rust Autodiff Tests LLVM 20-2] 🏁  Job succeeded

wsmoses · 2025-08-31T15:26:17Z

.github/workflows/enzyme-rust.yml

+      run: |
+        curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+        source ~/.cargo/env
+        rustup toolchain install nightly


until Rust adds CI for Enzyme, I think this should be fixed to a specific commit

.github/workflows/enzyme-rust.yml

wsmoses · 2025-08-31T15:27:10Z

.github/workflows/enzyme-rust.yml

+jobs:
+  rust-autodiff:
+    name: Rust Autodiff Tests LLVM ${{ matrix.llvm }}
+    runs-on: [self-hosted, Linux, X64, 32-core]


this is not the correct setup for the runner, see https://github.com/EnzymeAD/Enzyme/blob/main/.github/workflows/enzyme-mlir.yml for an example

it needs to run on linux-x86-n2-32

wsmoses · 2025-09-01T01:19:45Z

.github/workflows/enzyme-rust.yml

+        cmake ../enzyme -DLLVM_EXTERNAL_LIT=`which lit` -DCMAKE_BUILD_TYPE=Release -DLLVM_DIR=/usr/lib/llvm-${{ matrix.llvm }}/lib/cmake/llvm
+        make -j `nproc` LLVMEnzyme-${{ matrix.llvm }}
+    - name: Clone and configure Rust compiler
+      run: |


this is never told to use the enzyme we built above?

wsmoses · 2025-09-01T01:19:59Z

.github/workflows/enzyme-rust.yml

+        source ~/.cargo/env
+        git clone --depth 1 --branch master https://github.com/rust-lang/rust.git
+        cd rust
+        ./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs


we should also use ccache here, and cache intermediate artifacts

sgasho · 2025-09-01T15:43:17Z

@wsmoses @ZuseZ4
Thank you so much for the review!
However, I'm sorry for being a beginner at GitHub Actions and Enzyme and I'm not sure how to properly fix about some comments, especially

#2430 (comment)
#2430 (comment)
#2430 (comment)

Could you give me some hints, examples, suggestions, or something I should study?
I did some commits, but I'm not confident, especially wip

wsmoses · 2025-09-02T14:07:22Z

@ZuseZ4 can you help @sgasho understand the next steps from the comments?

ZuseZ4 · 2025-09-02T17:49:19Z

I have no experience with ccache and to me it sounds like something that should be added in a follow-up pr to avoid feature creep. But it looks like you've already added it in your last commit? If it works I guess it should stay.

For the pinning of the version, I would assume we want to fix the rustc we build, and not necessarily the rust version we use to build rustc. It's fine to do both, but right now you only pin the version used to build. You should also check out a specific commit of github.com/rust-lang/rust instead of just the master branch. This way, we can verify that new Enzyme versions don't break support. We would update the rustc commit every once in a while. This way, an update to rustc can't break Enzyme CI.

I don't know whats the relevant difference between your. yml file and the mlir example given, so I'd leave that for Billy to answer.

I-Al-Istannen · 2025-09-02T21:10:01Z

.github/workflows/enzyme-rust.yml

+        echo "CC=ccache gcc" >> $GITHUB_ENV
+        echo "CXX=ccache g++" >> $GITHUB_ENV


Who uses this? What exactly is built using GCC? I think Rust still uses LLVM. The rust invocation will prefix its invocations with ccache by itself as far as I can see, thanks to the --enable-ccache a few lines above.

.github/workflows/enzyme-rust.yml

I-Al-Istannen · 2025-09-02T21:14:04Z

.github/workflows/enzyme-rust.yml

+      run: |
+        . ~/.cargo/env
+        rm -rf build && mkdir build && cd build
+        RUST_LLVM_DIR=${GITHUB_WORKSPACE}/rust/build/x86_64-unknown-linux-gnu/llvm


I would shell-quote this. GitHub probably has no space or special chars in the checkout dir, but better be safe.

I shell-quoted it in this commit!
use quotes

I-Al-Istannen · 2025-09-02T21:19:01Z

.github/workflows/enzyme-rust.yml

+      run: |
+        curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+        . ~/.cargo/env
+        # Pin to specific nightly commit for reproducibility


What is this used for? Rustc bootstrap? Might be worth it to add a comment saying what is now reproducible. Enzyme itself is built against the latest rust compiler (you check out the main branch of rust-lang/rust), so I imagine it won't really help there?

wsmoses · 2025-09-03T14:44:12Z

.github/workflows/enzyme-rust.yml


    container:
-      image: ${{ (contains(matrix.os, 'linux') && 'ghcr.io/enzymead/reactant-docker-images@sha256:91e1edb7a7c869d5a70db06e417f22907be0e67ca86641d48adcea221fedc674' ) || '' }}
+      image: ${{ (contains(matrix.os, 'linux') && 'ghcr.io/enzymead/reactant-docker-images:main' ) || '' }}


this should use the specific hash, not main

wsmoses · 2025-09-03T18:32:10Z

.github/workflows/enzyme-rust.yml

+    - name: Install dependencies
+      run: |
+        apt-get update
+        apt-get install -y binutils ninja-build cmake gcc g++ python3 python3-dev ccache nodejs npm


do you need all these deps?

wsmoses · 2025-09-03T18:33:00Z

.github/workflows/enzyme-rust.yml

+        . ~/.cargo/env
+        git clone https://github.com/rust-lang/rust.git
+        cd rust
+        git checkout 51ff895062ba60a7cba53f57af928c3fb7b0f2f4


can you move the commit to the matrix? that'll make it easier to update, and or if in the future we want to checkout multiple and/or stable tags

ZuseZ4 · 2025-09-05T19:45:57Z

This currently builds Enzyme twice, once as part of rustc and once standalone.
Unfortunately it uses the old one from rustc for testing.
Is there a way to delete the default Enzyme submodule and re-add it using the new Enzyme source code which we have here (e.g. through ln -s or copying)? Then you can stop to build it seperately and we use the normal rust infra for building it.

.github/workflows/enzyme-rust.yml

wsmoses · 2025-09-08T13:01:02Z

.github/workflows/enzyme-rust.yml

+    - name: Clone and configure Rust compiler
+      run: |
+        . ~/.cargo/env
+        git clone https://github.com/rust-lang/rust.git


you should use another actions/checkout for this [and can give it the relevant tag/commit]

ZuseZ4 · 2025-09-10T17:06:56Z

@wsmoses this looks ready? Can you run CI, I don't have permission

wsmoses · 2025-09-20T22:28:58Z

.github/workflows/enzyme-rust.yml

+          --enable-clang \
+          --enable-lld \
+          --disable-optimize-llvm \
+          --enable-option-checking \


we should use the github actions cache here, but lets see if it builds first

otherwise ccache doesn't do anything as the run is ephemeral

wsmoses · 2025-09-20T23:26:01Z

fails

CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-gnu = None
CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu = None
HOST_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_PREFIX_PATH_x86_64-unknown-linux-gnu = None
CMAKE_PREFIX_PATH_x86_64_unknown_linux_gnu = None
HOST_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_x86_64-unknown-linux-gnu = None
CMAKE_x86_64_unknown_linux_gnu = None
HOST_CMAKE = None
CMAKE = None
running: cd "/__w/Enzyme/Enzyme/rust/build/x86_64-unknown-linux-gnu/enzyme/build" && CMAKE_PREFIX_PATH="" DESTDIR="" LC_ALL="C" LLVM_CONFIG_REAL="/__w/Enzyme/Enzyme/rust/build/x86_64-unknown-linux-gnu/llvm/bin/llvm-config" "cmake" "/__w/Enzyme/Enzyme/rust/src/tools/enzyme/enzyme/" "-G" "Ninja" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER_LAUNCHER=ccache" "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache" "-DCMAKE_C_COMPILER=cc" "-DCMAKE_CXX_COMPILER=c++" "-DCMAKE_ASM_COMPILER=cc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_SHARED_LINKER_FLAGS=" "-DCMAKE_MODULE_LINKER_FLAGS=" "-DCMAKE_EXE_LINKER_FLAGS=" "-DLLVM_ENABLE_ASSERTIONS=ON" "-DENZYME_EXTERNAL_SHARED_LIB=ON" "-DENZYME_BC_LOADER=OFF" "-DLLVM_DIR=/__w/Enzyme/Enzyme/rust/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_INSTALL_PREFIX=/__w/Enzyme/Enzyme/rust/build/x86_64-unknown-linux-gnu/enzyme" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Debug"
CMake Error: The source directory "/__w/Enzyme/Enzyme/rust/src/tools/enzyme/enzyme" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.

thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5:

ZuseZ4 · 2025-09-26T14:22:23Z

@sgasho Not sure if it's related, or needed but you can also try to do a submodule update for enzyme and commit it.

wsmoses · 2025-09-26T23:42:02Z

Run source ~/.cargo/env
Run '/home/runner/k8s/index.js'
using script executor
/__w/_temp/4f4ab6b8-84af-4b9c-8304-d6f25cfb5572.sh: 1: source: not found
Error: Error: ScriptExecutorError when trying to execute: "Error: Job failed with exit code 127."
Error: Process completed with exit code 1.
Error: Executing the custom container implementation failed. Please contact your self hosted runner admini

ZuseZ4 · 2025-10-01T20:30:18Z

@sgasho Looks like you managed it! The last test is failing, but that one frequently changes the IR generated, not sure if it's because of rustc, LLVM, or Enzyme changing. I'll look into making it more robust, it's not like that feature (batching) is well tested in Rust atm anyway.

From my side that sounds like it's good to merge?

Also side note, we'll probably switch from linking against Enzyme to just dlopen'ing Enzyme, which should make this CI even simpler in the future: rust-lang/rust#146623

wsmoses · 2025-10-01T20:40:37Z

while @ZuseZ4 fixes the last test to pass, @sgasho can you get the ccache stuff working?

I think those are the last two things in order to merge!

…youxu bless autodiff batching test This pr blesses a broken test and unblocks running rust in the Enzyme CI: EnzymeAD/Enzyme#2430 Enzyme is the plugin used by our std::autodiff and (future) std::batching modules, both of which are not build by default. In the near future we also hope to enable std::autodiff in the Rust CI. This test is the only one to combine two features, automatic differentiation and batching/vectorization. This combination is even more experimental than either feature on its own. I have a wip branch in which I enable more vectorization/batching and as part of that I'll think more about how to write those tests in a robust way (and likely change the interface). Until that lands, I don't care too much about what specific IR we generate here; it's just nice to track changes. r? compiler

sgasho · 2025-10-17T14:03:28Z

Does "Deleting the contents of '/__w/Enzyme/Enzyme'" below have to do with this low hit rate?
ccache is under /__w/Enzyme/Enzyme/.ccache, so cache might be deleted before the build step....
Placing the “Cache ccache” step after actions/checkout@v4 might resolve this issue?

ZuseZ4 · 2025-10-17T14:21:17Z

Not sure about that part.
@I-Al-Istannen mentioned though that it would be good to move the first show stats a little bit down, to run after the Cache ccache step where the cache gets restored.

ZuseZ4 · 2025-10-19T17:25:28Z

I'm not sure why it's still not caching, but at this point it's probably easier to wait for my fix on the rustc side to land.
Due to some past work std::autodiff/enzyme now also works with download-ci-llvm=true.
Download-LLVM optionally also ships with llvm assertions, so we don't loose anything.

rust-lang/rust#147888

ZuseZ4 · 2025-10-20T16:43:48Z

@sgasho The PR landed. If you update again and use the new build command from that PR it shouldn't build LLVM anymore. I'd also remove ccache now, since it seems unreliable. We should be fast enough without it.

wsmoses · 2025-10-21T22:53:57Z

I would still keep ccache, can you restore that?

giordano · 2025-10-22T10:54:54Z

https://github.com/EnzymeAD/Enzyme/actions/runs/18688055156/job/53367987961?pr=2430#step:8:460

thread 'main' (2614) panicked at src/bootstrap/src/core/config/config.rs:1768:22:
called `Result::unwrap()` on an `Err` value: "command did not execute successfully: cd \"/__w/Enzyme/Enzyme/rust\" && \"git\" \"rev-parse\" \"HEAD^1\"\nexpected success, got: exit status: 128\n"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

sgasho · 2025-10-22T14:50:28Z

thread 'main' (2614) panicked at src/bootstrap/src/core/config/config.rs:1768:22:
called Result::unwrap() on an Err value: "command did not execute successfully: cd "/__w/Enzyme/Enzyme/rust" && "git" "rev-parse" "HEAD^1"\nexpected success, got: exit status: 128\n"
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

hmm....I couldn't reproduce this on my local machine using act.
I explicitly set fetch-depth to 0 for now.

specify fetch-depth

sgasho · 2025-10-22T15:15:28Z

git rev-parse HEAD^1 problem has been resolved.
Another error occurred. I'll check it tomorrow.

-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
LLVM_SHLIBEXT=.so
LLVM ABS DIR /__w/Enzyme/Enzyme/rust/build/x86_64-unknown-linux-gnu/ci-llvm
CMAKE_PREFIX_PATH 
CMake Error at CMakeLists.txt:74 (find_package):
  Could not find a package configuration file provided by "LLVM" with any of
  the following names:

    LLVMConfig.cmake
    llvm-config.cmake

  Add the installation prefix of "LLVM" to CMAKE_PREFIX_PATH or set
  "LLVM_DIR" to a directory containing one of the above files.  If "LLVM"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!

thread 'main' (2652) panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5:

command did not execute successfully, got: exit status: 1

build script failed, must exit now
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
	finished in 0.289 seconds
Build completed unsuccessfully in 0:01:42
Error: Error: ScriptExecutorError when trying to execute: "Error: Job failed with exit code 1."
Error: Process completed with exit code 1.
Error: Executing the custom container implementation failed. Please contact your self hosted runner administrator.

giordano · 2025-10-22T22:21:32Z

.github/workflows/enzyme-rust.yml

+    - uses: actions/checkout@v4
+      name: Checkout Enzyme
+    - uses: actions/checkout@v4


Suggested change

- uses: actions/checkout@v4

name: Checkout Enzyme

- uses: actions/checkout@v4

- uses: actions/checkout@v5

name: Checkout Enzyme

- uses: actions/checkout@v5

ZuseZ4 · 2025-10-22T23:08:52Z

The second error looks familiar, https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/distribute.20LLVM.20CMake.20modules.3F/near/420114940
Hopefully, adding these files to an existing component is easy. Let's see what the infra people say.

ZuseZ4 · 2025-10-23T12:53:38Z

This PR could be a potential solution @sgasho @I-Al-Istannen.
I'm not super sure what's the best way to test it, I wrote an option in the PR but maybe you have a better idea?
rust-lang/rust#148027

ZuseZ4 · 2025-10-24T16:33:01Z

It looks like the cmake files alone are not enough and people are not keen on shipping additional things just for one user, so Enzyme should just build it's own LLVM. Even with a cache that barely has any cache hits, the last run was only 23 minutes once we remove cland and lld. That's already at the lower end of what the Julia runners need, so from a Rust point I'd recommend to land that and open an issue for someone to investigate why ccache has a low hitrate, to make it even faster.

wsmoses · 2025-10-24T16:38:54Z

What was the error message when cmake was shipped?

wsmoses · 2025-10-24T16:39:04Z

What was the error message when cmake was shipped?

ZuseZ4 · 2025-10-24T16:41:40Z

rust-lang/rust#148027 (comment), and looking at the corresponding CMake file it would have looked for all other archives next (if I read cmake correctly).

ZuseZ4 · 2025-10-24T22:50:20Z

We just added Rust CI to our Enzyme fork at rust-lang/enzyme.
It unfortunately doesn't use download-ci-llvm=true due to the issues mentioned above, but even on the slow public runners it finishes in 18 minutes, so on the better runners it should be significantly faster than some of the other langs and should not be a bottleneck. rust-lang@098e4da

@sgasho I'm not sure what's the difference and why the rust-lang/enzyme had a much higher cache hit rate, but maybe the PR helps you to figure out the difference? Can you try to update your pr (while still keeping the faster custom runners)?
Then we're hopefully able to get this pr landed soon!

ZuseZ4 · 2025-10-25T14:14:53Z

My guess is that you're missing the --enable-llvm-link-shared flag, which was in the last revision of the docs (which I guess I'll need to revert). Without it rust might build llvm as static, in which case it never generates an llvm.a file, but rather a lot of individual archives. Enzyme incorrectly tries to link llvm, and then fails.

I have a bugfix in the rust fork for it, which is likely why CI there doesn't need it. Upstreaming unfortunately seems blocked on some julia failure, hopefully someone will pick it up.

sgasho · 2025-10-27T13:53:40Z

32 minutes for the first run, without cache.
190MB cache has been created.

@wsmoses @giordano
Could you run it again? Now we can see how much faster it gets

sgasho · 2025-10-27T14:14:30Z

@giordano

Could you run it again? Now we can see how much faster it gets

Thanks!

The second run took 10 minutes, which was three times faster than without the cache.

wsmoses

lgtm, @giordano can you also give this a once over

giordano

Looks good to me 👍

sgasho · 2025-10-27T16:04:44Z

@wsmoses @giordano @ZuseZ4 @I-Al-Istannen
Thank you so much for your support!

I-Al-Istannen · 2025-10-27T17:15:42Z

Congrats on the merge and grit to keep at it :)

ZuseZ4 · 2025-10-27T17:23:58Z

Agree, thanks a lot for having the patience to work two months on it! Let's hope that we'll soon get ci (and thus nightly support) within rust as well.

wsmoses reviewed Aug 31, 2025

View reviewed changes

.github/workflows/enzyme-rust.yml Show resolved Hide resolved

wsmoses reviewed Aug 31, 2025

View reviewed changes

wsmoses reviewed Sep 1, 2025

View reviewed changes

I-Al-Istannen reviewed Sep 2, 2025

View reviewed changes

sgasho force-pushed the rust-autodiff-ci branch from 959f1ee to f6c53f9 Compare September 3, 2025 14:33

wsmoses reviewed Sep 3, 2025

View reviewed changes

sgasho force-pushed the rust-autodiff-ci branch from ec70a9d to 2071224 Compare September 4, 2025 15:58

sgasho commented Sep 7, 2025

View reviewed changes

.github/workflows/enzyme-rust.yml Outdated Show resolved Hide resolved

wsmoses reviewed Sep 8, 2025

View reviewed changes

sgasho force-pushed the rust-autodiff-ci branch from eadd4f3 to 9e3b7aa Compare September 11, 2025 14:42

ZuseZ4 requested a review from wsmoses September 20, 2025 22:25

wsmoses reviewed Sep 20, 2025

View reviewed changes

sgasho force-pushed the rust-autodiff-ci branch from 726c421 to 79282f3 Compare September 27, 2025 02:28

ZuseZ4 mentioned this pull request Oct 3, 2025

bless autodiff batching test rust-lang/rust#147315

Merged

giordano reviewed Oct 22, 2025

View reviewed changes

ci: add ci to test rust std::autodiff

bf4d02d

sgasho force-pushed the rust-autodiff-ci branch from 8ba359e to bf4d02d Compare October 26, 2025 12:51

Merge branch 'main' into rust-autodiff-ci

7e4bfc9

wsmoses approved these changes Oct 27, 2025

View reviewed changes

giordano approved these changes Oct 27, 2025

View reviewed changes

wsmoses merged commit aecafad into EnzymeAD:main Oct 27, 2025
26 checks passed

ZuseZ4 mentioned this pull request Oct 27, 2025

Add std::autodiff and Rust to the Enzyme CI rust-lang/rust#145899

Open

BrewTestBot mentioned this pull request Oct 30, 2025

enzyme 0.0.206 Homebrew/homebrew-core#251926

Merged

		echo "CC=ccache gcc" >> $GITHUB_ENV
		echo "CXX=ccache g++" >> $GITHUB_ENV

ci: add a github action ci to test Rust std::autodiff #2430

ci: add a github action ci to test Rust std::autodiff #2430

Uh oh!

Conversation

sgasho commented Aug 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgasho commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wsmoses commented Sep 2, 2025

Uh oh!

ZuseZ4 commented Sep 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZuseZ4 commented Sep 5, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZuseZ4 commented Sep 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wsmoses commented Sep 20, 2025

Uh oh!

ZuseZ4 commented Sep 26, 2025

Uh oh!

wsmoses commented Sep 26, 2025

Uh oh!

ZuseZ4 commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wsmoses commented Oct 1, 2025

Uh oh!

sgasho commented Oct 17, 2025

Uh oh!

ZuseZ4 commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZuseZ4 commented Oct 19, 2025

Uh oh!

ZuseZ4 commented Oct 20, 2025

Uh oh!

wsmoses commented Oct 21, 2025

Uh oh!

giordano commented Oct 22, 2025

Uh oh!

sgasho commented Oct 22, 2025

Uh oh!

sgasho commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sgasho commented Aug 31, 2025 •

edited

Loading

sgasho commented Sep 1, 2025 •

edited

Loading

ZuseZ4 commented Oct 1, 2025 •

edited

Loading

ZuseZ4 commented Oct 17, 2025 •

edited

Loading

sgasho commented Oct 22, 2025 •

edited

Loading

ZuseZ4 commented Oct 25, 2025 •

edited

Loading

sgasho commented Oct 27, 2025 •

edited

Loading

I-Al-Istannen commented Oct 27, 2025 •

edited

Loading