[CI][Github] Install Clang in Windows container #164519

cmtice · 2025-10-21T23:25:31Z

Downloads clang-for-windows from the LLVM releases website, decompresses and untars the images, and leave them in C:\clang\clang-msvc... Temporarily downloads the 'xz' utility to decompress the downloaded clang tarball image.

Downloads clang-for-windows from the LLVM releases website, decompresses and untars the images, and leave them in C:\clang\clang-msvc\... Temporarily downloads the 'xz' utility to decompress the downloaded clang tarball image.

llvmbot · 2025-10-21T23:26:13Z

@llvm/pr-subscribers-github-workflow

Author: None (cmtice)

Changes

Downloads clang-for-windows from the LLVM releases website, decompresses and untars the images, and leave them in C:\clang\clang-msvc... Temporarily downloads the 'xz' utility to decompress the downloaded clang tarball image.

Full diff: https://github.com/llvm/llvm-project/pull/164519.diff

1 Files Affected:

(modified) .github/workflows/containers/github-action-ci-windows/Dockerfile (+34)

diff --git a/.github/workflows/containers/github-action-ci-windows/Dockerfile b/.github/workflows/containers/github-action-ci-windows/Dockerfile
index 9ddf5017bc020..7835cc07a3936 100644
--- a/.github/workflows/containers/github-action-ci-windows/Dockerfile
+++ b/.github/workflows/containers/github-action-ci-windows/Dockerfile
@@ -98,3 +98,37 @@ RUN powershell -Command \
     Add-Type -AssemblyName System.IO.Compression.FileSystem ; \
     [System.IO.Compression.ZipFile]::ExtractToDirectory('actions-runner-win.zip', $PWD) ;\
     rm actions-runner-win.zip
+
+# Download and extract Clang compiler.
+# Create directories, download, extract, and clean up all in one layer
+RUN powershell -Command \
+    # --- Setup directories --- \
+    Write-Host "Creating directories..."; \
+    New-Item -Path "C:\temp-download" -ItemType "Directory" -Force ; \
+    New-Item -Path "C:\xz-utils" -ItemType "Directory" -Force ; \
+    New-Item -Path "C:\clang" -ItemType "Directory" -Force ; \
+    # --- 1. Download and extract xz --- \
+    Set-Location C:\temp-download ; \
+    Write-Host "Downloading xz-utils..."; \
+    Invoke-WebRequest -Uri "https://github.com/tukaani-project/xz/releases/download/v5.8.1/xz-5.8.1-windows.zip" -OutFile "xz.zip"; \
+    Write-Host "Extracting xz-utils..."; \
+    Add-Type -AssemblyName "System.IO.Compression.FileSystem"; \
+    [System.IO.Compression.ZipFile]::ExtractToDirectory('C:\temp-download\xz.zip', 'C:\xz-utils'); \ 
+    # --- 2. Download and decompress Clang --- \
+    Write-Host "Downloading Clang..."; \
+    Invoke-WebRequest -Uri "https://github.com/llvm/llvm-project/releases/download/llvmorg-21.1.2/clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar.xz" -OutFile "clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar.xz" ; \
+    Write-Host "Decompressing clang.tar.xz using C:\xz-utils\bin_x86-64\xz.exe"; \
+    C:\xz-utils\bin_x86-64\xz.exe -d -qq clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar.xz ; \
+    # --- 3. Extract clang --- \
+    Write-Host "Extracting clang.tar to C:\clang ..."; \
+    C:\Windows\System32\tar.exe -xf clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar -C C:\clang ; \
+    # --- 4. Clean up --- \
+    Write-Host "Cleaning up..." ; \
+    Set-Location C:\ ; \
+    Remove-Item C:\temp-download -Recurse -Force; \
+    Remove-Item C:\xz-utils -Recurse -Force; \
+    Write-Host "Download and extraction complete." ;
+
+RUN powershell -Command \
+    Set-Location C:\clang ; \
+    Rename-Item -Path "C:\clang\clang+llvm-21.1.2-x86_64-pc-windows-msvc" -NewName "C:\clang\clang-msvc" ;

boomanaiden154

How does the size of this image compare to the existing one? docker image ls should show unpacked image sizes.

.github/workflows/containers/github-action-ci-windows/Dockerfile

boomanaiden154 · 2025-10-21T23:35:50Z

.github/workflows/containers/github-action-ci-windows/Dockerfile

+    # --- 1. Download and extract xz --- \
+    Set-Location C:\temp-download ; \
+    Write-Host "Downloading xz-utils..."; \
+    Invoke-WebRequest -Uri "https://github.com/tukaani-project/xz/releases/download/v5.8.1/xz-5.8.1-windows.zip" -OutFile "xz.zip"; \


Is it possible to verify a checksum here? Similar to what we do for sccache in the linux container:

llvm-project/.github/workflows/containers/github-action-ci/Dockerfile

Line 77 in 93dd17a

echo $( [ $(arch) = 'x86_64' ] && echo "1fbb35e135660d04a2d5e42b59c7874d39b3deb17de56330b25b713ec59f849b" || echo "d6a1ce4acd02b937cd61bc675a8be029a60f7bc167594c33d75732bbc0a07400") /tmp/sccache.tar.gz | sha256sum -c && \

Done (I think).

boomanaiden154 · 2025-10-21T23:36:14Z

.github/workflows/containers/github-action-ci-windows/Dockerfile

+    [System.IO.Compression.ZipFile]::ExtractToDirectory('C:\temp-download\xz.zip', 'C:\xz-utils'); \ 
+    # --- 2. Download and decompress Clang --- \
+    Write-Host "Downloading Clang..."; \
+    Invoke-WebRequest -Uri "https://github.com/llvm/llvm-project/releases/download/llvmorg-21.1.2/clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar.xz" -OutFile "clang+llvm-21.1.2-x86_64-pc-windows-msvc.tar.xz" ; \


Can we pull the version number here into an ENV line?

llvm-project/.github/workflows/containers/github-action-ci/Dockerfile

Line 5 in 93dd17a

ENV LLVM_VERSION=21.1.3

boomanaiden154 · 2025-10-21T23:36:52Z

.github/workflows/containers/github-action-ci-windows/Dockerfile

+
+RUN powershell -Command \
+    Set-Location C:\clang ; \
+    Rename-Item -Path "C:\clang\clang+llvm-21.1.2-x86_64-pc-windows-msvc" -NewName "C:\clang\clang-msvc" ;


This is the binary? Why are we renaming it to clang-msvc?

No, that's not the binary. That's the directory into which tar untar'd everything. I'm shortening the path name. With this renaming, the clang-cl binary is in C:\clang\clang-msvc\bin\clang-cl.exe I thought the very long path name would be a pain to type and a likely source of typos.

cmtice · 2025-10-21T23:43:30Z

How does the size of this image compare to the existing one? docker image ls should show unpacked image sizes.

26.1 GB vs 17.7 GB, I think.

boomanaiden154 · 2025-10-21T23:45:20Z

26.1 GB vs 17.7 GB, I think.

As in the new one is ~9GB smaller? That doesn't make a lot of sense.

cmtice · 2025-10-21T23:49:01Z

26.1 GB vs 17.7 GB, I think.

As in the new one is ~9GB smaller? That doesn't make a lot of sense.

No, the new one is ~9GB bigger. Sorry that wasn't clear in my previous answer. New image: 26.1 GB; old image: 17.7 GB.

cmtice · 2025-10-21T23:52:28Z

26.1 GB vs 17.7 GB, I think.

As in the new one is ~9GB smaller? That doesn't make a lot of sense.

No, the new one is ~9GB bigger. Sorry that wasn't clear in my previous answer. New image: 26.1 GB; old image: 17.7 GB.

Note: The downloaded clang includes a LOT more than just the clang-cl.exe file:

C:>ls clang\clang-msvc
bin include lib libexec share

C:>ls clang\clang-msvc\bin
LLVM-C.dll llvm-cxxmap.exe
LTO.dll llvm-debuginfo-analyzer.exe
Remarks.dll llvm-debuginfod-find.exe
amdgpu-arch.exe llvm-debuginfod.exe
analyze-build llvm-diff.exe
bugpoint.exe llvm-dis.exe
c-index-test.exe llvm-dlltool.exe
clang++.exe llvm-dwarfdump.exe
clang-apply-replacements.exe llvm-dwarfutil.exe
clang-change-namespace.exe llvm-dwp.exe
clang-check.exe llvm-exegesis.exe
clang-cl.exe llvm-extract.exe
clang-cpp.exe llvm-gsymutil.exe
clang-doc.exe llvm-ifs.exe
clang-extdef-mapping.exe llvm-install-name-tool.exe
clang-format.exe llvm-jitlink.exe
clang-include-cleaner.exe llvm-lib.exe
clang-include-fixer.exe llvm-libtool-darwin.exe
clang-installapi.exe llvm-link.exe
clang-linker-wrapper.exe llvm-lipo.exe
clang-move.exe llvm-lto.exe
clang-nvlink-wrapper.exe llvm-lto2.exe
clang-offload-bundler.exe llvm-mc.exe
clang-offload-packager.exe llvm-mca.exe
clang-query.exe llvm-ml.exe
clang-refactor.exe llvm-ml64.exe
clang-reorder-fields.exe llvm-modextract.exe
clang-repl.exe llvm-mt.exe
clang-scan-deps.exe llvm-nm.exe
clang-sycl-linker.exe llvm-objcopy.exe
clang-tblgen.exe llvm-objdump.exe
clang-tidy.exe llvm-opt-report.exe
clang.exe llvm-otool.exe
clangd.exe llvm-pdbutil.exe
diagtool.exe llvm-profdata.exe
dsymutil.exe llvm-profgen.exe
find-all-symbols.exe llvm-ranlib.exe
git-clang-format llvm-rc.exe
git-clang-format.bat llvm-readelf.exe
hmaptool llvm-readobj.exe
intercept-build llvm-readtapi.exe
ld.lld.exe llvm-reduce.exe
ld64.lld.exe llvm-remarkutil.exe
libclang.dll llvm-rtdyld.exe
libiomp5md.dll llvm-sim.exe
liblldb.dll llvm-size.exe
libomp.dll llvm-split.exe
llc.exe llvm-stress.exe
lld-link.exe llvm-strings.exe
lld.exe llvm-strip.exe
lldb-argdumper.exe llvm-symbolizer.exe
lldb-dap.exe llvm-tblgen.exe
lldb-instr.exe llvm-tli-checker.exe
lldb-server.exe llvm-undname.exe
lldb.exe llvm-windres.exe
lli.exe llvm-xray.exe
llvm-addr2line.exe modularize.exe
llvm-ar.exe nvptx-arch.exe
llvm-as.exe offload-arch.exe
llvm-bcanalyzer.exe opt.exe
llvm-bitcode-strip.exe pp-trace.exe
llvm-c-test.exe reduce-chunk-list.exe
llvm-cat.exe run-clang-tidy
llvm-cfi-verify.exe sancov.exe
llvm-cgdata.exe sanstats.exe
llvm-config.exe scan-build
llvm-cov.exe scan-build-py
llvm-ctxprof-util.exe scan-build.bat
llvm-cvtres.exe scan-view
llvm-cxxdump.exe verify-uselistorder.exe
llvm-cxxfilt.exe wasm-ld.exe

boomanaiden154 · 2025-10-22T00:01:58Z

Note: The downloaded clang includes a LOT more than just the clang-cl.exe file:

We should try and trim what we extract as much as possible. We need only one or two of those executables, so the rest just pessimize image unpack times significantly given the 50% size increase. I would suspsect we only need to keep clang, lld (although maybe the windows versions of those executables), and some resource headers.

- Remove "Write-Host" lines - Add LLVM_VERSION environment variable - Remove unnecessary files from downloaded clang

cmtice · 2025-10-23T14:55:07Z

I've updated this to address most review comments; I have not added the md5 checksum yet; will get to that soon.

I updated the container to remove many of the downloaded files from clang; I kept the entire include directory, and the entire lib directory. In the bin directory I kept clang-cl.exe and lld-link.exe.

The old container size was 17.7 GB. The new container size is 23.2 GB. This is not making a lot of sense to me -- I checked the size of the stuff I kept from the new clang download, and it's only a total of 1.2 GB (bin: 469MB; lib:665MB; include: 78MB). I entered the container and checked and the new stuff did indeed all get deleted, so I have no idea where the other new 4.3 GB came from.

boomanaiden154 · 2025-10-23T15:01:14Z

The old container size was 17.7 GB. The new container size is 23.2 GB. This is not making a lot of sense to me -- I checked the size of the stuff I kept from the new clang download, and it's only a total of 1.2 GB (bin: 469MB; lib:665MB; include: 78MB). I entered the container and checked and the new stuff did indeed all get deleted, so I have no idea where the other new 4.3 GB came from.

You need to delete the files in the same layer (RUN step) that you download them in. Otherwise you'll end up with one layer downloading them and then another layer deleting them, and both layers get preserved due to how docker works.

cmtice · 2025-10-23T17:10:44Z

The old container size was 17.7 GB. The new container size is 23.2 GB. This is not making a lot of sense to me -- I checked the size of the stuff I kept from the new clang download, and it's only a total of 1.2 GB (bin: 469MB; lib:665MB; include: 78MB). I entered the container and checked and the new stuff did indeed all get deleted, so I have no idea where the other new 4.3 GB came from.

You need to delete the files in the same layer (RUN step) that you download them in. Otherwise you'll end up with one layer downloading them and then another layer deleting them, and both layers get preserved due to how docker works.

Ah, much better! When I do that, the new container size is 19.1 GB.

.github/workflows/containers/github-action-ci-windows/Dockerfile

Downloads clang-for-windows from the LLVM releases website, decompresses and untars the images, and leave them in C:\clang\clang-msvc\... Temporarily downloads the 'xz' utility to decompress the downloaded clang tarball image.

[CI][Github] Install Clang in Windows container

8dc19da

Downloads clang-for-windows from the LLVM releases website, decompresses and untars the images, and leave them in C:\clang\clang-msvc\... Temporarily downloads the 'xz' utility to decompress the downloaded clang tarball image.

cmtice requested a review from boomanaiden154 October 21, 2025 23:25

llvmbot added the github:workflow label Oct 21, 2025

boomanaiden154 reviewed Oct 21, 2025

View reviewed changes

Address some reviewer comments:

dfc1898

- Remove "Write-Host" lines - Add LLVM_VERSION environment variable - Remove unnecessary files from downloaded clang

Put cleaning up clang download in same layer as download event.

7015843

Add md5 checksum checks on the downloaded files.

4a58bdd

boomanaiden154 reviewed Oct 23, 2025

View reviewed changes

.github/workflows/containers/github-action-ci-windows/Dockerfile Outdated Show resolved Hide resolved

Use Docker ENV keyword to define LLVM_VERSION.

39db1d2

boomanaiden154 approved these changes Oct 23, 2025

View reviewed changes

cmtice merged commit 2b808c0 into llvm:main Oct 23, 2025
12 of 13 checks passed

Uh oh!

[CI][Github] Install Clang in Windows container #164519

[CI][Github] Install Clang in Windows container #164519

Uh oh!

Conversation

cmtice commented Oct 21, 2025

Uh oh!

llvmbot commented Oct 21, 2025

Uh oh!

boomanaiden154 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

boomanaiden154 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

cmtice Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

boomanaiden154 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

cmtice Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

boomanaiden154 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

cmtice Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

cmtice commented Oct 21, 2025

Uh oh!

boomanaiden154 commented Oct 21, 2025

Uh oh!

cmtice commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmtice commented Oct 21, 2025

Uh oh!

boomanaiden154 commented Oct 22, 2025

Uh oh!

cmtice commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boomanaiden154 commented Oct 23, 2025

Uh oh!

cmtice commented Oct 23, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cmtice commented Oct 21, 2025 •

edited

Loading

cmtice commented Oct 23, 2025 •

edited

Loading