Switch to the new ARM64 Helix queues #77764

MichalStrehovsky · 2022-11-02T04:07:37Z

Want to check if this works, no merge for now.

Cc @dotnet/runtime-infrastructure

dotnet-issue-labeler · 2022-11-02T04:07:44Z

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

MichalStrehovsky · 2022-11-02T04:08:13Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-11-02T04:08:27Z

Azure Pipelines successfully started running 1 pipeline(s).

ghost · 2022-11-02T06:17:10Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

Want to check if this works, no merge for now.

Cc @dotnet/runtime-infrastructure

Author:	MichalStrehovsky
Assignees:	MichalStrehovsky
Labels:	`area-Infrastructure`
Milestone:	-

MattGal · 2022-11-02T15:32:07Z

Consider trying ubuntu.2004.armarch / ubuntu.2004.armarch.open as well.

MichalStrehovsky · 2022-11-02T20:29:55Z

@dotnet/area-system-globalization this is switching the tests from running on Windows 10 ARM64 to Windows 11 ARM64. Is it expected that these tests would fail? Do they need special OS configuration? They're failing both in the regular libraries run and NativeAOT libraries run on ARM64.

[FAIL] System.Globalization.Tests.NumberFormatInfoCurrencyGroupSizes.CurrencyGroupSizes_Get_ReturnsExpected(format: NumberFormatInfo { }, expected: [3, 2])
Assert.Equal() Failure
              ? (pos 1)
Expected: [3, 2]
Actual:   [3]
   at System.Globalization.Nls.Tests!<BaseAddress>+0x89e4b4
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.CultureInfoConstructor.Ctor_String(name: "prs", expectedNames: ["prs"], expectToThrowOnBrowser: True)
Assert.Contains() Failure
Not found: fa
In value:  String[] ["prs"]
   at System.Globalization.Tests.CultureInfoConstructor.Ctor_String(String name, String[] expectedNames, Boolean expectToThrowOnBrowser) + 0xa0
   at System.Globalization.Nls.Tests!<BaseAddress>+0x8a7760
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.CultureInfoConstructor.Ctor_String(name: "prs-AF", expectedNames: ["prs-AF"], expectToThrowOnBrowser: True)
Assert.Contains() Failure
Not found: fa-AF
In value:  String[] ["prs-AF"]
   at System.Globalization.Tests.CultureInfoConstructor.Ctor_String(String name, String[] expectedNames, Boolean expectToThrowOnBrowser) + 0xa0
   at System.Globalization.Nls.Tests!<BaseAddress>+0x8a7760
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.NumberFormatInfoMiscTests.DigitSubstitutionListTest(cultureName: "prs", shape: NativeNational)
Assert.Equal() Failure
Expected: Context
Actual:   NativeNational
   at System.Globalization.Tests.NumberFormatInfoMiscTests.DigitSubstitutionListTest(String cultureName, DigitShapes shape) + 0x54
   at System.Globalization.Nls.Tests!<BaseAddress>+0x89e604
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c

MattGal · 2022-11-02T20:32:01Z

@dotnet/area-system-globalization this is switching the tests from running on Windows 10 ARM64 to Windows 11 ARM64. Is it expected that these tests would fail? Do they need special OS configuration? They're failing both in the regular libraries run and NativeAOT libraries run on ARM64.

[FAIL] System.Globalization.Tests.NumberFormatInfoCurrencyGroupSizes.CurrencyGroupSizes_Get_ReturnsExpected(format: NumberFormatInfo { }, expected: [3, 2])
Assert.Equal() Failure
              ? (pos 1)
Expected: [3, 2]
Actual:   [3]
   at System.Globalization.Nls.Tests!<BaseAddress>+0x89e4b4
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.CultureInfoConstructor.Ctor_String(name: "prs", expectedNames: ["prs"], expectToThrowOnBrowser: True)
Assert.Contains() Failure
Not found: fa
In value:  String[] ["prs"]
   at System.Globalization.Tests.CultureInfoConstructor.Ctor_String(String name, String[] expectedNames, Boolean expectToThrowOnBrowser) + 0xa0
   at System.Globalization.Nls.Tests!<BaseAddress>+0x8a7760
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.CultureInfoConstructor.Ctor_String(name: "prs-AF", expectedNames: ["prs-AF"], expectToThrowOnBrowser: True)
Assert.Contains() Failure
Not found: fa-AF
In value:  String[] ["prs-AF"]
   at System.Globalization.Tests.CultureInfoConstructor.Ctor_String(String name, String[] expectedNames, Boolean expectToThrowOnBrowser) + 0xa0
   at System.Globalization.Nls.Tests!<BaseAddress>+0x8a7760
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c
[FAIL] System.Globalization.Tests.NumberFormatInfoMiscTests.DigitSubstitutionListTest(cultureName: "prs", shape: NativeNational)
Assert.Equal() Failure
Expected: Context
Actual:   NativeNational
   at System.Globalization.Tests.NumberFormatInfoMiscTests.DigitSubstitutionListTest(String cultureName, DigitShapes shape) + 0x54
   at System.Globalization.Nls.Tests!<BaseAddress>+0x89e604
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x14c

I saw this. Typically it's because it's the newest windows we've tested on and @tarekgh magically shows up and makes the tests understand the new world and/or fixes the bugs.

tarekgh · 2022-11-02T21:16:18Z

@MattGal is correct, Windows can change the underlying globalization data. But I expect this test will fail on all Windows 11 and not only arm64 except if the Windows 11 arm64 somehow carries different data. But, in general this shouldn't block the PR as it is most likely a globalization data or test issue.

MattGal · 2022-11-02T22:09:27Z

@tarekgh one thing to note here is that currently the only Windows 11 ARM SKU in Azure is a "preview" SKU; it's slightly newer than the AMD64 versions.

MichalStrehovsky · 2022-11-25T06:55:47Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-11-25T06:56:06Z

Azure Pipelines successfully started running 1 pipeline(s).

MichalStrehovsky · 2022-12-20T08:37:54Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-12-20T08:38:10Z

Azure Pipelines successfully started running 1 pipeline(s).

MattGal · 2022-12-20T18:44:44Z

Looks like the issues with System.Globalization.Extensions.Nls.Tests continue. Just wanted to add here that it might be time to disable this test on Windows 11 til it can be investigated and fixed to match the changing OS conditions, since we'll be rolling out changes to forward the deprecated Windows 10 ARM queues to the Windows 11 versions next month (approximately 1/11/2023)

MichalStrehovsky · 2022-12-21T00:52:14Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-12-21T00:52:28Z

Azure Pipelines successfully started running 1 pipeline(s).

MichalStrehovsky · 2022-12-21T00:54:02Z

Looks like the issues with System.Globalization.Extensions.Nls.Tests continue. Just wanted to add here that it might be time to disable this test on Windows 11 til it can be investigated and fixed to match the changing OS conditions, since we'll be rolling out changes to forward the deprecated Windows 10 ARM queues to the Windows 11 versions next month (approximately 1/11/2023)

Yeah, I was hoping we wouldn't have to do that, but maybe blocking them just on ARM64 Windows is still fine. We still get results from x64 Windows and chances for an arch-specific bug cropping in globalization while it's disabled are pretty low.

Pushed out a commit that blocks them.

MichalStrehovsky · 2022-12-21T03:00:06Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-12-21T03:00:30Z

Azure Pipelines successfully started running 1 pipeline(s).

MichalStrehovsky · 2022-12-21T21:59:25Z

Looks good to merge. The failures in the runtime legs are all known per build analysis. Runtime-extra-platforms has a lot of failures but it's par for the course. None of the legs are Arm64 Windows.

ghost assigned MichalStrehovsky Nov 2, 2022

MichalStrehovsky added the area-Infrastructure label Nov 2, 2022

MattGal approved these changes Nov 2, 2022

View reviewed changes

MichalStrehovsky force-pushed the queue branch from fc0076f to c661dd6 Compare November 25, 2022 06:55

build-analysis bot mentioned this pull request Nov 25, 2022

CI build failure: Build MacCatalyst x64 Release AllSubsets_Mono - XcodeBuildApp task failed unexpectedly - Could not find System.Runtime.Tests.app #78778

Closed

Switch to the new ARM64 Helix queues

1b72ff7

MichalStrehovsky force-pushed the queue branch from c661dd6 to 1b72ff7 Compare December 20, 2022 08:37

build-analysis bot mentioned this pull request Dec 20, 2022

Precondition failure: File has not had execution verified #79439

Closed

runfoapp bot mentioned this pull request Dec 20, 2022

system.text.regularexpressions.tests work item #72834

Open

MichalStrehovsky mentioned this pull request Dec 21, 2022

Some culture tests failing on latest Windows 11 builds #79867

Closed

Block the failing tests

f9b83e6

Missed one

0853274

build-analysis bot mentioned this pull request Dec 21, 2022

emcc received SIGKILL #79874

Closed

MichalStrehovsky merged commit 8d66372 into dotnet:main Dec 21, 2022

MichalStrehovsky deleted the queue branch December 21, 2022 21:59

ghost locked as resolved and limited conversation to collaborators Jan 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to the new ARM64 Helix queues #77764

Switch to the new ARM64 Helix queues #77764

MichalStrehovsky commented Nov 2, 2022

dotnet-issue-labeler bot commented Nov 2, 2022

MichalStrehovsky commented Nov 2, 2022

azure-pipelines bot commented Nov 2, 2022

ghost commented Nov 2, 2022

MattGal commented Nov 2, 2022

MichalStrehovsky commented Nov 2, 2022

MattGal commented Nov 2, 2022

tarekgh commented Nov 2, 2022

MattGal commented Nov 2, 2022

MichalStrehovsky commented Nov 25, 2022

azure-pipelines bot commented Nov 25, 2022

MichalStrehovsky commented Dec 20, 2022

azure-pipelines bot commented Dec 20, 2022

MattGal commented Dec 20, 2022

MichalStrehovsky commented Dec 21, 2022

azure-pipelines bot commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022

azure-pipelines bot commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022

Switch to the new ARM64 Helix queues #77764

Switch to the new ARM64 Helix queues #77764

Conversation

MichalStrehovsky commented Nov 2, 2022

dotnet-issue-labeler bot commented Nov 2, 2022

MichalStrehovsky commented Nov 2, 2022

azure-pipelines bot commented Nov 2, 2022

ghost commented Nov 2, 2022

MattGal commented Nov 2, 2022

MichalStrehovsky commented Nov 2, 2022

MattGal commented Nov 2, 2022

tarekgh commented Nov 2, 2022

MattGal commented Nov 2, 2022

MichalStrehovsky commented Nov 25, 2022

azure-pipelines bot commented Nov 25, 2022

MichalStrehovsky commented Dec 20, 2022

azure-pipelines bot commented Dec 20, 2022

MattGal commented Dec 20, 2022

MichalStrehovsky commented Dec 21, 2022

azure-pipelines bot commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022

azure-pipelines bot commented Dec 21, 2022

MichalStrehovsky commented Dec 21, 2022