Checksum verification failed on Windows 2019 and Windows 2022

[Ami-007]

Description

Attempted to build both Windows 2019 and Windows 2022 images based on the current repo, but they are both failing with the error below:

==> azure-arm: Checksum verification failed: expected 2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374c7306a1015ec83d300c1092f11f3869
==> azure-arm: 58686b70431cec109257ace163a79551f5be797fb8b6a0ba, got A48E753165286E567507C2C73ABF3C38BB0C50015352424851ADCEE75DD884564
==> azure-arm: C29DC66DC82346E62CD95BDED680D5A6722D12253088B1E6717C0FDE8570042
==> azure-arm: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:919 char:9
==> azure-arm: + throw "Checksum verification failed: expected $expectedHash, ...
==> azure-arm: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm: + CategoryInfo : OperationStopped: (Checksum verifi...717C0FDE8570042:String) [], RuntimeException
==> azure-arm: + FullyQualifiedErrorId : Checksum verification failed: expected 2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374
==> azure-arm: c7306a1015ec83d300c1092f11f386958686b70431cec109257ace163a79551f5be797fb8b6a0ba, got A48E753165286E567507C2C73ABF3
==> azure-arm: C38BB0C50015352424851ADCEE75DD884564C29DC66DC82346E62CD95BDED680D5A6722D12253088B1E6717C0FDE8570042

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 11
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

Private builds of Azure DevOps agents based on Windows Server 2019 and Windows Server 2022 from this repo

Is it regression?

Yes

Expected behavior

Checksums match and builds proceeds successfully.

Actual behavior

==> azure-arm: Provisioning with powershell script: /agent/_work/1/s/runner-images/images/windows/scripts/build/Install-DotnetSDK.ps1
azure-arm: Downloading package from https://dot.net/v1/dotnet-install.ps1 to C:\Users\packer\AppData\Local\Temp\dotnet-install.ps1...
azure-arm: Package downloaded in 0.81 seconds
azure-arm: Downloading package from https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json to C:\Users\packer\AppData\Local\Temp\releases.json...
azure-arm: Package downloaded in 0.56 seconds
azure-arm: Installing dotnet 6.0.131
azure-arm: dotnet-install: Remote file https://dotnetcli.azureedge.net/dotnet/Sdk/6.0.131/dotnet-sdk-6.0.131-win-x64.zip size is 251514428 bytes.
azure-arm: dotnet-install: Downloaded file https://dotnetcli.azureedge.net/dotnet/Sdk/6.0.131/dotnet-sdk-6.0.131-win-x64.zip size is 251514428 bytes.
azure-arm: dotnet-install: The remote and local file sizes are equal.
azure-arm: dotnet-install: Extracting the archive.
azure-arm: dotnet-install: Adding to current process PATH: "C:\Program Files\dotnet". Note: This change will not be visible if PowerShell was run as a child process.
azure-arm: dotnet-install: Note that the script does not resolve dependencies during installation.
azure-arm: dotnet-install: To check the list of dependencies, go to https://learn.microsoft.com/dotnet/core/install/windows#dependencies
azure-arm: dotnet-install: Installed version is 6.0.131
azure-arm: dotnet-install: Installation finished
azure-arm: Downloading package from https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json to C:\Users\packer\AppData\Local\Temp\releases.json...
azure-arm: Package downloaded in 0.13 seconds
==> azure-arm: Checksum verification failed: expected 2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374c7306a1015ec83d300c1092f11f3869
==> azure-arm: 58686b70431cec109257ace163a79551f5be797fb8b6a0ba, got A48E753165286E567507C2C73ABF3C38BB0C50015352424851ADCEE75DD884564
==> azure-arm: C29DC66DC82346E62CD95BDED680D5A6722D12253088B1E6717C0FDE8570042
==> azure-arm: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:919 char:9
==> azure-arm: + throw "Checksum verification failed: expected $expectedHash, ...
==> azure-arm: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm: + CategoryInfo : OperationStopped: (Checksum verifi...717C0FDE8570042:String) [], RuntimeException
==> azure-arm: + FullyQualifiedErrorId : Checksum verification failed: expected 2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374
==> azure-arm: c7306a1015ec83d300c1092f11f386958686b70431cec109257ace163a79551f5be797fb8b6a0ba, got A48E753165286E567507C2C73ABF3
==> azure-arm: C38BB0C50015352424851ADCEE75DD884564C29DC66DC82346E62CD95BDED680D5A6722D12253088B1E6717C0FDE8570042
==> azure-arm:

Repro steps

Build the Windows Server 2019 or Windows Server 2022 image using the code in runner images

[ihari277]

I am also receiving the same error. Could someone please verify and update this?

=> azure-arm.image: Checksum verification failed: expected 2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374c7306a1015ec83d300c1092f11f3869
==> azure-arm.image: 58686b70431cec109257ace163a79551f5be797fb8b6a0ba, got A48E753165286E567507C2C73ABF3C38BB0C50015352424851ADCEE75DD884564
==> azure-arm.image: C29DC66DC82346E62CD95BDED680D5A6722D12253088B1E6717C0FDE8570042

[pbcahill]

Same. I temporarily worked around it by commenting out the lines doing that checksum verification in the Install-DotnetSDK.ps1 script:

[LubovKrasikova]

The same issue with running a build locally for win-19 server to update Chrome Browser to v125 from:
https://github.com/actions/runner-images/releases/tag/win19%2F20240526.1

and from successful main build:

[yo1-markenstein]

I notice that the Install-DotnetSDK.ps1 downloads the install package from here: https://dotnetcli.azureedge.net/dotnet/Sdk/6.0.131/dotnet-sdk-6.0.131-win-x64.zip (by using https://dot.net/v1/dotnet-install.ps1).
Then it checks the hash code with the corresponding value obtained from https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json:

           {
              "name": "dotnet-sdk-win-x64.zip",
              "rid": "win-x64",
              "url": "https://download.visualstudio.microsoft.com/download/pr/ce77d3f6-4248-463d-931c-30f67208dd2c/ee2e8a08f2e490028d58450cf40a9cca/dotnet-sdk-6.0.131-win-x64.zip",
              "hash": "2655938bc0e94e788b847c45adcc87a8c4afd160294a0f374c7306a1015ec83d300c1092f11f386958686b70431cec109257ace163a79551f5be797fb8b6a0ba"
            },

These are different sources. The file downloaded with the url in the releases.json is consistent with the hash.
Apparently there is a mismatch between the package hosted by https://dotnetcli.azureedge.net and the package hosted in https://download.visualstudio.microsoft.com.

[vamshisiram]

I am getting the same issue. can anyone please help on this?

[kishorekumar-anchala]

HI @vamshisiram , we're working on your request , we will share the latest update ASAP.

[vamshisiram]

Thanks for the acknowledgment @kishorekumar-anchala

[vamshisiram]

Same. I temporarily worked around it by commenting out the lines doing that checksum verification in the Install-DotnetSDK.ps1 script:

@kishorekumar-anchala the workaround suggested by @pbcahill worked for me. Just wanted to let you know, in case it helps troubleshooting @kishorekumar-anchala

[kishorekumar-anchala]

Hi @Ami-007

Could you please try to re-deploy by forking the main repo , confirm back if same error occurs.

[Ami-007]

Hi @kishorekumar-anchala,

Retrying now, we will know in about 4-5 hours.

Thanks

[yo1-markenstein]

I just successfully completed the build using the main branch.
The checksum issue seem to be solved: the package downloaded from https://dotnetcli.azureedge.net results in the hash code that matches https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json.
No code changes (or workarounds) needed.

[vamshisiram]

This is very good news. Thanks Kishore for taking care of this. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Johan Markenstein ***@***.***> Sent: Friday, May 31, 2024 6:33:27 AM To: actions/runner-images ***@***.***> Cc: vamshisiram ***@***.***>; Mention ***@***.***> Subject: Re: [actions/runner-images] Checksum verification failed on Windows 2019 and Windows 2022 (Issue #9958) I just successfully completed the build using the main branch. The checksum issue seem to be solved: the package downloaded from https://dotnetcli.azureedge.net<https://dotnetcli.azureedge.net/> results in the hash code that matches https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json. No code changes (or workarounds) needed. — Reply to this email directly, view it on GitHub<#9958 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVU36WS6QVJQLSVOCF2PIPDZFB32PAVCNFSM6AAAAABIPMITUSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBSGE3TMMBWGQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ami-007]

I just successfully completed the build using the main branch. The checksum issue seem to be solved: the package downloaded from https://dotnetcli.azureedge.net results in the hash code that matches https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/6.0/releases.json. No code changes (or workarounds) needed.

When did you run a build? We are now seeing a signature thumbprint error when packer is installing AWSTools. @kishorekumar-anchala we can’t check if the checksum is working due to this issue now as pipeline is failing there instead

[yo1-markenstein]

@Ami-007 : my message concerns only the "Checksum verification failed on Windows 2019 and Windows 2022" Issue.
Not related to anything AWSTools related.