Add containerized test legs (Azure Linux, Alpine/musl, Nano Server) - #4515
Merged
Conversation
Runs the SkiaSharp console test suite inside a container using the repo's
existing bootstrapper `docker:` feature — the same compliant mechanism the
native Linux builds use (build the env image with the approved container-image
task, then run `dotnet cake --target=<target>` against the mounted repo).
No bespoke multi-stage Dockerfiles, no wrapper script, no raw docker build.
- scripts/infra/tests/tests-container.cake + build.cake 'tests-container' target:
runs ONLY the console suite (no externals build). Builds with
-p:TargetFrameworks=net10.0 to avoid the Android workload, and selects the
native build under test with --nativePlatform=<linux|alpine|nanoserver>.
- Real native swap (not an overlay): --nativePlatform flows to the
SkiaSharp/HarfBuzzSharp NativePlatform MSBuild property in
binding/IncludeNativeAssets.*.targets, so the build copies ONLY that
platform's library. This is what lets the Windows container run the
nanoserver lib and Alpine run the musl lib (neither is mapped by the
OS-derived defaults). Empty property = unchanged default behavior.
- scripts/infra/tests/docker/{glibc,alpine,nanoserver}/Dockerfile: environment
images (Linux glibc + fonts; Alpine/musl + fonts; Nano Server .NET SDK with
no fonts — the env under test).
- azure-templates-jobs-bootstrapper.yml: extend the docker feature to support
Windows containers (branch shell/mount by agent OS; only pass
--platform linux/amd64 on non-Windows). Native docker legs unaffected.
- azure-templates-stages-test.yml: tests_container_linux,
tests_container_alpine_linux, tests_container_nanoserver_windows legs —
docker: env image + target: tests-container, consuming the `native` artifact
(never externals-download), publishing the TRX. Enabled by default,
non-gating. Intentionally do NOT flow use1ESPipelineTemplates (tests never
run on 1ES). TRX goes to the standard ADO Tests tab via PublishTestResults.
- documentation/dev/containerized-testing.md: full design + real proofs.
Proven end-to-end locally via the actual feature commands (build env image,
docker run cake target on the mounted repo):
- Linux glibc: 5902 tests, 5720 passed, 182 skipped, 0 failed; the swap put
ONLY the linux .so in the output dir (no .dll/.dylib).
- Alpine musl: 5715 passed, 182 skipped, 5 failed — 4 are missing-Symbola-font
environment gaps on Alpine, 1 is a known GC-timing flake; exactly the kind of
per-platform result these non-gating legs exist to catalogue.
Nano Server is the same path on a Windows container host (cannot run on this
macOS host).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Rewrite documentation/dev/containerized-testing.md and the related YAML/cake/ targets comments to describe what the feature is and how it works, without narrating prior iterations, contrasts with removed approaches, or transient test results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Contributor
📦 Try the packages from this PRWarning Do not run these scripts without first reviewing the code in this PR. Step 1 — Download the packages bash / macOS / Linux: curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4515PowerShell / Windows: iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4515"Step 2 — Add the local NuGet source dotnet nuget add source ~/.skiasharp/hives/pr-4515/packages --name skiasharp-pr-4515More options
Or download manually from Azure Pipelines — look for the Remove the source when you're done: dotnet nuget remove source skiasharp-pr-4515 |
Contributor
|
📖 Documentation Preview The documentation for this PR has been deployed and is available at: 🔗 View Staging Site This preview will be updated automatically when you push new commits to this PR. This comment is automatically updated by the documentation staging workflow. |
Alpine's font-dejavu doesn't cover the emoji (U+1F680) the unicode font tests need, and Alpine has no Symbola package. Install font-noto-emoji (family "Noto Color Emoji") in the Alpine env image, and pick UnicodeFontFamilies per libc (Symbola on glibc, Noto Color Emoji on musl) via PlatformConfiguration.IsGlibc. Explain the fontconfig vs non-fontconfig font behavior in the docs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Contributor
📊 SkiaSharp benchmarks — PR #4515⭐ this PR (full source build) vs 🌙 nightly · Linux · Windows · macOS
Highlights⏱️ Time — 🔴 54 slower · 🟢 24 faster
Full per-OS benchmark deltasLinux⏱️ Time (vs 🌙 nightly
Windows⏱️ Time (vs 🌙 nightly
macOS⏱️ Time (vs 🌙 nightly
|
Contributor
📦 Artifact size reportPackages from this PR (build Total Packages
+38 package(s) unchanged (< 50.0 KB). Per-file changes
|
| File | Size |
|---|---|
runtimes/win-x86/native/libHarfBuzzSharp.pdb |
22.2 MB → 21.9 MB (🟢 −272.0 KB) |
runtimes/win-x64/native/libHarfBuzzSharp.pdb |
22.0 MB → 21.8 MB (🟢 −240.0 KB) |
runtimes/win-arm64/native/libHarfBuzzSharp.pdb |
22.3 MB → 22.3 MB (🟢 −8.0 KB) |
SkiaSharp.NativeAssets.WinUI
| File | Size |
|---|---|
runtimes/win-arm64/native/libGLESv2.pdb |
64.6 MB → 64.3 MB (🟢 −272.0 KB) |
runtimes/win-x86/native/libGLESv2.pdb |
67.3 MB → 67.0 MB (🟢 −256.0 KB) |
runtimes/win-x64/native/libGLESv2.pdb |
66.6 MB → 66.4 MB (🟢 −240.0 KB) |
runtimes/win-x64/native/libEGL.pdb |
6.4 MB → 6.3 MB (🟢 −32.0 KB) |
runtimes/win-x86/native/libEGL.pdb |
6.5 MB → 6.5 MB (🟢 −16.0 KB) |
runtimes/win-arm64/native/SkiaSharp.Views.WinUI.Native.pdb |
67.7 MB → 67.7 MB (🟢 −8.0 KB) |
runtimes/win-x64/native/SkiaSharp.Views.WinUI.Native.pdb |
68.0 MB → 68.0 MB (🟢 −8.0 KB) |
🧬 win-x64 (runtimes/win-x64/native/libGLESv2.dll) |
4.9 MB → 4.9 MB (🔴 +2.0 KB) |
🧬 win-arm64 (runtimes/win-arm64/native/libGLESv2.dll) |
4.4 MB → 4.4 MB (🔴 +1.0 KB) |
Informational only — this never blocks the PR. Native binaries are labelled by os/arch.
Tests that need enumerable system fonts are tagged
[Trait("Category", "RequiresSystemFonts")]. The container legs whose native
build has no enumerable system fonts (linuxnodeps/alpinenodeps, built
skia_use_fontconfig=false, and Nano Server) pass --skipSystemFonts, which
forwards --filter-not-trait "Category=RequiresSystemFonts" to the runner.
- tests/Tests/Traits.cs: add the RequiresSystemFonts category.
- tag the 30 font-dependent methods (39 cases with theory data) across
SKFontManagerTest, SKFontStyleSetTest, SKFontTest, SKPaintTest, SKTypefaceTest.
- tests-container.cake: add --skipSystemFonts; test-shared.cake RunDotNetTest
gains an extraRunnerArgs passthrough.
- azure-templates-stages-test.yml: add Linux NoDeps + Alpine NoDeps legs and
pass --skipSystemFonts=true on the fontless legs (interim filter, removable
from the legs once the tests self-skip on a fontless environment).
Verified in-container: linuxnodeps without the filter fails 39 font tests; with
--skipSystemFonts it excludes exactly those 39 and is green (5681 passed, 182
skipped, 0 failed).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Replaces the interim [Trait("Category","RequiresSystemFonts")] + --skipSystemFonts
approach with runtime capability probes, so the same suite runs unmodified
everywhere and each platform skips what it genuinely can't do.
Two capabilities, probed at runtime (BaseTest):
- SkipWhenNoSystemFontManager(): no manager to enumerate/match fonts — WASM
(no manager), NoDependencies Linux builds and Nano Server (empty manager,
MatchCharacter returns null). Applied to the SKFontManager/SKFontStyleSet and
MatchCharacter (emoji) tests; supersedes their prior IsBrowser-only skips.
- SkipWhenNoDefaultFont(): default typeface is empty — NoDependencies and Nano
Server (SKTypeface.Default.IsEmpty). Applied to the measure/break/draw tests
that use the default typeface. WASM keeps its single embedded default font, so
these keep running there.
Reverts the RequiresSystemFonts trait, the --skipSystemFonts cake switch and its
RunDotNetTest passthrough, and the --skipSystemFonts=true leg args.
Capability matrix confirmed by probing each build (SKFontManager.Default /
SKTypeface.Default): fontconfig linux/alpine have real fonts; linuxnodeps and
alpinenodeps report one empty family, an empty default typeface and null
MatchCharacter. Verified in-container: linuxnodeps self-skips and is green
(5681 passed, 221 skipped, 0 failed) with no filter; linux still runs the font
tests unchanged (5720 passed, 182 skipped).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
The Windows container run now writes each command on its own line to cmd.cmd and prints it (Get-Content), then runs that file in the container — the same self-printing script-file pattern as the Linux cmd.sh path, instead of an inline 'cmd /c a && b'. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Remove continueOnError from the container legs so a failure fails the build. If these platforms are worth building and testing, a regression should block, not be silently ignored. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
The CI Linux agents are x64, so the base image already resolves to linux/amd64 by default; the explicit flag was a carryover from local arm64 testing and is a no-op on CI. Removing it collapses the per-OS ternary to a plain tag argument. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new CI/testing capability to run SkiaSharp.Tests.Console inside Docker containers against selected prebuilt native outputs, extending coverage to container-only environments (Linux glibc, Alpine/musl, and Windows Nano Server) and enabling explicit native platform selection during managed builds.
Changes:
- Introduces a
tests-containerCake target plus container environment Dockerfiles, and wires new Azure Pipelines legs to run/publish TRX results. - Adds MSBuild native-asset overrides (
SkiaSharpNativePlatform/HarfBuzzSharpNativePlatform) so container jobs can selectoutput/native/<platform>/<arch>/explicitly. - Updates font-related tests/config to be more container/font-manager aware (glibc vs musl font family selection; runtime skips for fontless environments).
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Tests/SkiaSharp/SKTypefaceTest.cs | Adjusts emoji/system-font test skipping behavior for container/fontless environments. |
| tests/Tests/SkiaSharp/SKPaintTest.cs | Skips a pixel-assert text test when no default font exists (fontless containers). |
| tests/Tests/SkiaSharp/SKFontTest.cs | Adds font-manager/default-font skips to stabilize tests across container/fontless environments. |
| tests/Tests/SkiaSharp/SKFontStyleSetTest.cs | Replaces browser-only skips with “no system font manager” detection. |
| tests/Tests/SkiaSharp/SKFontManagerTest.cs | Replaces browser-only skips with “no system font manager” detection. |
| tests/Tests/BaseTest.cs | Adds shared helpers for detecting “no system font manager” / “no default font” and skipping accordingly. |
| tests/SkiaSharp.Tests.Console/DefaultTestConfig.Console.cs | Selects Linux unicode font family based on glibc vs musl (Symbola vs Noto Color Emoji). |
| scripts/infra/tests/tests-container.cake | New Cake script to build+run only the console tests in a container against prebuilt natives. |
| scripts/infra/tests/docker/nanoserver/Dockerfile | New Windows Nano Server SDK env image for running console tests in a fontless Windows container. |
| scripts/infra/tests/docker/glibc/Dockerfile | New Linux glibc SDK env image with required font packages. |
| scripts/infra/tests/docker/alpine/Dockerfile | New Alpine/musl SDK env image with required runtime deps + fonts. |
| scripts/azure-templates-stages-test.yml | Adds new containerized test legs and publishes TRX/artifacts. |
| scripts/azure-templates-jobs-bootstrapper.yml | Extends docker: bootstrapper feature to run on Windows agents (cmd + Windows volume mount). |
| documentation/dev/README.md | Adds dev docs link for containerized testing. |
| documentation/dev/containerized-testing.md | New documentation describing how containerized test legs work and how to run locally. |
| build.cake | Adds tests-container target entrypoint. |
| binding/IncludeNativeAssets.SkiaSharp.targets | Adds SkiaSharpNativePlatform override to copy natives from output/native/<platform>/<arch>/. |
| binding/IncludeNativeAssets.HarfBuzzSharp.targets | Adds HarfBuzzSharpNativePlatform override to copy natives from output/native/<platform>/<arch>/. |
Comments suppressed due to low confidence (2)
tests/Tests/SkiaSharp/SKFontTest.cs:103
- The SKTypeface returned by SKFontManager.Default.MatchCharacter should be disposed. Other tests use
using varfor this call; leaving it undisposed can leak native handles in long test runs.
var emojiChar = StringUtilities.GetUnicodeCharacterCode(text, SKTextEncoding.Utf32);
var typeface = SKFontManager.Default.MatchCharacter(emojiChar);
Assert.NotNull(typeface);
using var font = new SKFont();
font.Typeface = typeface;
tests/Tests/SkiaSharp/SKTypefaceTest.cs:249
- The SKTypeface returned by SKFontManager.Default.MatchCharacter should be disposed (it owns a native handle). Use
using varhere to avoid leaking the typeface across the test run.
SkipWhenNoSystemFontManager();
const string text = "🚀";
var emojiChar = StringUtilities.GetUnicodeCharacterCode(text, SKTextEncoding.Utf32);
var typeface = SKFontManager.Default.MatchCharacter(emojiChar);
Assert.NotNull(typeface);
The doc still described the container legs as non-gating with continueOnError; the legs are now gating and rely on the runtime self-skip helpers for fontless environments. Correct the description. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
The Nano Server .NET SDK image has no sn.exe, so the Windows-only _SignAssembly / _SignAssemblyVerify targets failed the container build with exit code 9009 before any test could run. Gate both targets behind a new DisableStrongNameSigning property and set it from the tests-container cake target. PublicSign still applies, so the assemblies keep the public key identity that InternalsVisibleTo requires (same as the Linux legs, where these targets never run). Default behavior is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
With the sn.exe re-sign skipped, the Nano leg progressed to the next Windows-only build step: MDocGenerateDocs runs mdoc.exe, which cannot start on Nano Server (exit -1073741515 / STATUS_DLL_NOT_FOUND — missing Windows DLL deps). Doc generation is irrelevant to running the test suite, so set the existing SkipMDocGenerateDocs opt-out for all container builds. These are the only two external-tool build steps gated on IsWindows+Release, so no further whack-a-mole expected. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Build 158986 failed due to a lost tvOS test agent (infra flake); the still-queued container legs were skipped as collateral and never ran. Empty commit to get a clean run for the Nano Server leg verification. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
The first full Nano Server container run passed 5687/220 with six real failures, all understood environment differences rather than harness bugs. Address each properly: XPS (SKDocumentTest) — Nano has no XPS Object Model / DirectWrite, so CreateXps returns null even though it IsWindows. Split the three tests that branched on IsWindows into straight-line supported tests (gated by a SupportsXps capability probe) plus one CreateXpsReturnsNullWhereUnsupported test for the null contract on Nano and non-Windows. System fonts (SKTypefaceTest / SKFontManagerTest GCStillCollectsTypeface) — both resolve 'Times New Roman' / C:\Windows\Fonts\times.ttf, absent on fontless Nano. Add SkipWhenNoSystemFontManager alongside the existing Windows-path guard. Visual text (VisualMatrixTests raster/Text) — Nano rasterizes text with FreeType, not DirectWrite, so it needs its own golden. Give Nano a 'nanoserver' golden tag that falls back to the shared 'windows' golden for cells that render identically (shapes), and commit the harvested raster.nanoserver/Text.png. Only divergent cells need a nano golden. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Replace the create-and-check-null XPS probe with a real platform indicator: TestConfig.IsNanoServer reads the registry InstallationType value, which is 'Nano Server' on Nano and 'Client'/'Server'/'Server Core' elsewhere. The registry read is Windows-guarded and isolated in a non-inlined method so the Microsoft.Win32 types are only JIT-resolved on Windows, never on the mobile/WASM test hosts (the type is framework- provided, so no package reference is needed). XPS support is now the declarative IsWindows && !IsNanoServer, and the visual golden 'nanoserver' tag keys off IsNanoServer directly instead of inferring it from an empty font manager. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
SupportsXps is a platform capability, so it belongs alongside IsWindows / IsNanoServer in BaseTest rather than as a private field on SKDocumentTest. Drop the singular VisualPlatform.Tag convenience accessor and use the plural Tags everywhere: Tags[0] for the primary write key (golden/diff marker paths, GoldenStore.Key) and the full Tags list in the unseeded- golden message so it names every candidate platform. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Fold the two-branch DetermineTags into one yield iterator: prepend the 'nanoserver' tag when on Nano Server, then yield the base OS tag from DetermineTag (which is 'windows' on Nano). This drops the duplicated 'windows' literal so the shared tag has a single source, while keeping OS identity (DetermineTag) separate from the golden-fallback policy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
Three review-driven robustness fixes:
- TestConfig.ReadIsNanoServer wraps the registry read in try/catch. A
constrained Windows host can deny the read (SecurityException, etc.),
and because the result is cached via Lazy<bool> that would re-throw on
every IsNanoServer access and fail the whole test config. Treat any
failure as 'not Nano Server'.
- The native-platform override in IncludeNativeAssets.{SkiaSharp,
HarfBuzzSharp}.targets now mirrors the existing OS-derived selection:
use $(Platform) when it is set and not AnyCPU, else $(BuildArch).
Previously it always used $(BuildArch), which could copy the wrong
arch (e.g. x64 instead of x86) if the override were ever used with an
explicit Platform. No behavior change for the container legs (they
build AnyCPU), but it removes the inconsistency with the default logic.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
…arer names Rework the Linux container test images so each leg tests what it claims, and rename the test-job display names to lead with the platform. Images: - Replace the glibc container image (Ubuntu) with Azure Linux. The desktop Linux leg already runs on Ubuntu, so this exercises a different, minimal glibc distro instead of a second Ubuntu run. - Give the No Dependencies legs their own bare images (azurelinux-nodeps, alpine-nodeps) that install NOTHING beyond the base .NET SDK. Previously they ran in the font-equipped images, which defeated the point: the leg exists to prove the nodeps native build (skia_use_fontconfig=false) works with no optional system packages. Verified the native libs load with only what the base image already provides. - Trim the Alpine image: drop bash, libstdc++, libgcc and icu-libs. Those are all already in the base .NET SDK image (the native library never needed them installed separately); only the fonts remain. Fonts: - Azure Linux ships no emoji font in any repo, so its Dockerfile downloads Noto Emoji (SIL OFL), pinned by URL commit and verified by SHA-256, for the Unicode tests. The glibc UnicodeFontFamilies list accepts both Symbola (Ubuntu desktop) and Noto Emoji (Azure Linux). Harness: - Run the in-container suite via /bin/sh instead of /bin/bash so the bare images need no shell package; the generated script is POSIX. Naming: display names now lead with the platform (macOS, Linux, Android, iOS, WASM), with brackets only for real variations — Windows (.NET Framework/.NET Core), WASM (.NET Preview), Linux (Azure Linux[, No Dependencies]) / (Alpine[, No Dependencies]), Windows (Nano Server). Validated locally: Azure Linux 5718 passed / 0 failed / 185 skipped with the downloaded emoji font resolving U+1F680; bare nodeps images load the native lib clean and self-skip the font tests; trimmed Alpine resolves fonts and runs with no bash. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
mattleibow
changed the title
The Linux (Azure Linux, No Dependencies) and Linux (Alpine, No Dependencies) container legs cover it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
mattleibow
pushed a commit
that referenced
this pull request
Jul 23, 2026
The new Alpine container legs intermittently killed the whole test run: the leg would stall on "2 tests running" and MTP's hangdump force-killed it (exit 7), or the process took SIGSEGV inside GPU shader compilation. It was flaky — CI retries sometimes passed — which made it look like a machine blip rather than a real fault. Root cause: on Linux the GL context comes from GLX via libX11, and the container has no X server. Normally XOpenDisplay(NULL) fails fast and CreateGlContext()'s catch skips the test. But running that from parallel GPU tests races the native GLX/GL init on musl: instead of failing fast it intermittently hangs or crashes in downstream GPU SkSL compilation. glibc fails fast reliably; musl does not, so only the Alpine legs flaked. It is not a regression in this PR's code — the PR merely added the Alpine legs that first exercise this path under load. Fix: skip in CreateGlContext() before touching any native code when there is no display (IsLinux and DISPLAY unset). The desktop Linux leg runs these under Xvfb with DISPLAY set, so its GPU coverage is unchanged; macOS/Windows use CGL/WGL rather than X11 and are unaffected. On headless Linux these tests already skipped in the common case, so no coverage is lost — only the racy native path is now never entered. Verified in-container (parallel + hangdump): both Alpine legs green with zero failures and no hang — alpine full 5718 passed / 185 skipped, alpine nodeps 5679 passed / 224 skipped. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
The new Alpine (No Dependencies) leg intermittently failed with the leg stalling on "2 tests running" until MTP's hangdump force-killed it (exit code 7) — reported as flaky since CI retries sometimes squeaked through. Root cause: SKBitmapThreadingTest.ImageScalingMultipleThreadsTest, the (100 threads, 1000 iterations) variant. It intentionally creates ~100K undisposed native allocations to stress GC-finalizer throughput (#3608) and already self-skips on x86, where "the GC can't keep up and Skia's native allocator fails". On x64 musl the same pressure across 100 threads stalls the allocator/finalizer interaction and the run hangs indefinitely instead of failing — deterministic once reproduced against the real CI x64 musl native lib (glibc keeps up and passes, which is why the existing glibc legs never hit this). It is not caused by this PR's code: the PR merely adds the first musl test legs, so this musl-only stress path is exercised here for the first time. Extend the existing x86 self-skip to also cover musl for the heavy variant. The lighter (10,10) and (10,100) variants still run everywhere, so concurrent-scaling coverage — and the #3608 regression guard — remains on musl. Add IsGlibc/IsMusl to TestConfig and IsMusl to BaseTest for the guard (musl == Linux and not glibc, via the existing gnu_get_libc_version probe already used for font selection). Verified against the real CI x64 musl nodeps native lib under x86-64 emulation: the heavy variant now skips (0 ms) and the full suite completes with no hang; and natively on arm64 musl the full suite is green (0 failed, 5678 passed, 225 skipped). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 611d76bf-f08f-44ab-85de-681c29a3bda9
mattleibow
force-pushed
the
mattleibow-nano-server-ci-tests
branch
from
July 23, 2026 20:54
0b09270 to
04c143c
Compare
This was referenced Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Runs the SkiaSharp console test suite (
SkiaSharp.Tests.Console) inside a Docker container against a chosen native build, covering container runtimes the agent-based test legs don't:linuxlinuxnodepsalpinealpinenodepsnanoserverIt reuses the repo's existing bootstrapper
docker:feature — the same mechanism the native Linux builds use: build an env image with theDocker@2task, then rundotnet cake --target=tests-containeragainst the mounted repo. Native libraries come from the mergednativepipeline artifact — nothing is downloaded or rebuilt in the leg. The legs are gating: a genuine failure fails the build.How
tests-containercake target (scripts/infra/tests/tests-container.cake+build.cake) — builds only the console suite with-p:TargetFrameworks=net10.0(single-TFM, so no Android/iOS workloads), then runs it. Builds no externals.--nativePlatform=<platform>sets aSkiaSharpNativePlatform/HarfBuzzSharpNativePlatformMSBuild property inbinding/IncludeNativeAssets.*.targets, so the build copies only the library fromoutput/native/<platform>/<arch>/(mirroring the existing OS-derived arch selection:$(Platform)when set, else host arch). This is how the Windows container runs thenanoserverbuild and Alpine runs themuslbuild (neither is mapped by the OS-derived defaults). Empty property = unchanged default behavior.scripts/infra/tests/docker/{azurelinux,azurelinux-nodeps,alpine,alpine-nodeps,nanoserver}/Dockerfile. Each image provisions only what its scenario needs (see Images below).docker:feature now runs on Windows agents too, branching the run step's shell and mount path by agent OS. The Linux path runs the suite via/bin/sh(POSIX, so the bare images need no shell package); the Windows path writes and self-prints acmd.cmdmirroring the Linuxcmd.sh.sn.exeand can't runmdoc.exe, both Windows-only build steps. The container build setsDisableStrongNameSigning(gates thesn -Rre-sign;PublicSignstill gives the correct identity forInternalsVisibleTo) andSkipMDocGenerateDocs(doc generation is irrelevant to running tests).Images
The glibc container uses Azure Linux rather than a second Ubuntu run (the desktop
Linuxleg already runs on Ubuntu) — a different, minimal glibc distro.The No Dependencies legs run in bare images that install nothing beyond the base .NET SDK. That is the point of those legs: they exercise the
skia_use_fontconfig=falsenative build against zero optional system packages, and fail iflibSkiaSharpever grows an undeclared dependency. The base SDK images already provide everything the native library links against (libstdc++,libgcc, libc, ICU), so nothing is installed — and the Alpine (full) image was trimmed to fonts only for the same reason.Fonts
The
nodepsand Nano Server builds have no enumerable system fonts (skia_use_fontconfig=false/ the custom-empty font manager), so font-dependent tests self-skip at runtime rather than fail. Two probes inBaseTestcapture the two distinct cases:SkipWhenNoSystemFontManager()— no manager to enumerate or match fonts (SKFontManager.Default.MatchCharacter('a')is null; guardsIsBrowserfirst). Covers WASM, thenodepsbuilds, and Nano Server.SkipWhenNoDefaultFont()— no usable default typeface (SKTypeface.Default.IsEmpty). Covers thenodepsbuilds and Nano Server, but not WASM (which keeps a single embedded default font).For the fontconfig legs: Alpine installs
font-noto-emoji(packaged). Azure Linux has no emoji font in any repo, so its Dockerfile downloads Noto Emoji (SIL OFL), pinned by URL commit and verified by SHA-256, for the Unicode tests (covers U+1F680). The glibcUnicodeFontFamilieslist accepts bothSymbola(Ubuntu desktop) andNoto Emoji(Azure Linux).Nano Server test fixes
The first full Nano run surfaced six failures, all understood environment differences (not harness bugs), each addressed properly:
SKDocumentTest) — Nano has no XPS Object Model / DirectWrite, soCreateXpsreturns null even though itIsWindows. Split the three tests that branched onIsWindowsinto straight-line supported tests plus oneCreateXpsReturnsNullWhereUnsupportedfor the null contract, gated bySupportsXps = IsWindows && !IsNanoServer.SKTypefaceTest/SKFontManagerTestGCStillCollectsTypeface) — both resolveTimes New Roman/C:\Windows\Fonts\times.ttf, absent on fontless Nano; addSkipWhenNoSystemFontManageralongside the existing Windows-path guard.VisualMatrixTestsraster/Text) — Nano rasterizes text with FreeType, not DirectWrite, so it gets ananoservergolden tag that falls back to the sharedwindowsgolden for cells that render identically (shapes). Committed the harvestedraster.nanoserver/Text.png.Nano is detected via
TestConfig.IsNanoServer, which reads the registryInstallationTypevalue ("Nano Server") — Windows-guarded, wrapped in try/catch (a constrained host can deny the read), and isolated so the Win32 types are only resolved on Windows.Job display names
Test-job display names now lead with the platform, with brackets only for real variations:
Windows (.NET Framework)/Windows (.NET Core)/Windows (Nano Server),macOS,Linux/Linux (Azure Linux[, No Dependencies])/Linux (Alpine[, No Dependencies]),Android,iOS,Mac Catalyst,WASM/WASM (.NET Preview).Validation
All five container legs run and pass in CI (Nano Server: 5689 passed / 0 failed / 225 skipped). Locally: Azure Linux 5718 passed / 0 failed / 185 skipped with the downloaded emoji font resolving U+1F680; the bare
nodepsimages load the native library clean and self-skip the font tests; the trimmed Alpine image resolves fonts and runs with no bash.Musl stress-test skip
Adding the first musl (Alpine) legs surfaced a latent hang in
SKBitmapThreadingTest.ImageScalingMultipleThreadsTest— the(100 threads, 1000 iterations)variant, which deliberately creates ~100K undisposed native allocations to stress GC-finalizer throughput (#3608). It already self-skips on x86 (where "the GC can't keep up and Skia's native allocator fails"); on x64 musl the same pressure stalls the allocator/finalizer interaction and the run hangs until hangdump kills the leg. The existing self-skip is extended to musl (glibc keeps up and still runs it), so the lighter(10,10)/(10,100)variants — and the #3608 regression guard — keep running there. Reproduced deterministically against the real CI x64 musl native lib and fixed.