[EXPERIMENT] Lock-striped HandleDictionary prototype (#4101)

[mattleibow]

⚗️ DRAFT / EXPERIMENT — not for merge. Opened to answer one question: does lock-striping the HandleDictionary have any measurable value worth pursuing? Answer below: conditional yes, and the original deadlock blocker is now resolved by a reservation-gate construction redesign.

Stacked on #4080 (base = dev/issue-3817-fix-singleton-init, so this diff shows only the striping change). Tracks #4101.

What this does

Replaces the single global HandleDictionary.instancesLock + single Dictionary with N shards, each its own Dictionary + IPlatformLock (+ DEBUG stackTraces). A handle is routed to exactly one shard via a golden-ratio bit-mix:

shard = (ulong)handle * 0x9E3779B97F4A7C15 >> 32 & (N-1)

• N = next-pow2(Environment.ProcessorCount) clamped [1, 64]. N=1 is byte-for-byte today's behavior (safety fallback + benchmark baseline).
• Customization is additive only (no ABI change — IPlatformLock is public/locked): AppContext.GetData("SkiaSharp.HandleDictionary.ShardCount") → env SKIASHARP_HANDLE_SHARDS → auto. Read once at first use.
• PlatformLock.Factory() is called N times (one lock per shard) — the existing plug-in contract is preserved unchanged.
• instances / stackTraces are now aggregating views over all shards so GarbageCleanupFixture and diagnostics keep working.

Why it might help

Every wrapper construction (GetOrAddObject) and disposal (SKObject.Dispose) takes the registry lock. Under heavy multi-threaded creation/teardown of distinct objects, that single lock serializes everything. Sharding lets disjoint handles proceed in parallel.

Benchmark — does it actually help? (12-core, Server GC, 7-rep median)

Churn of distinct handles, with realistic outside-lock work (50 spins/op):

threads	N=1	N=8	N=16	N=32	speedup
1	1.6	1.6	1.6	1.6	1.0x (zero overhead)
8	2.4	5.4	6.2	6.4	~2.6x
12	2.1	6.1	7.1	7.7	~3.5x
16	2.1	5.8	6.5	7.0	~3.2x

Knee ≈ ProcessorCount. Golden-ratio mix spreads aligned pointers perfectly (CoV ≈ 0%).

Counter-signal — shared-singleton workload (all threads hit ~8 shared handles):

threads	N=1	N=8	N=16
2	90.8	51.9	72.5
4	95.1	43.8	49.1

Here striping hurts (0.5–0.8x): contention is on a few handles that can't be spread, and the extra locks just bounce cache lines. The hot singletons (SKColorSpace.Srgb, etc.) are exactly this case.

Honest verdict

• ✅ Real win (2.5–3.7x) for high-thread parallel churn of distinct objects.
• ✅ Zero single-thread cost — safe default.
• ❌ No benefit (slight harm) for shared-singleton hot paths.
• ⚠️ Microbench overstates lock weight; real Skia ctor/dtor cost dilutes the win. An integration benchmark is needed before believing real apps hit this bottleneck.

This is "worth a guarded experiment," not a slam-dunk.

✅ Cross-shard deadlock blocker — RESOLVED (reservation-gate two-phase construction)

The first cut of this prototype ran the wrapper factory under the shard lock. With one global lock a factory that constructs a second, different-handle SKObject simply re-enters the same lock; with striping that second object may hash to a different shard, so a thread holding shard A grabs shard B while another holding B wants A → a classic AB-BA cross-shard deadlock that cannot occur today.

Commit 28c63c1 removes this structurally by reworking GetOrAddObject so the factory never runs under any shard lock:

1. Phase 1 (under shard lock, O(1)) — dedup against a live instance; else if another thread is already building this handle, wait on its per-handle gate and retry; else install our own Reservation and release the lock.
2. Phase 2 (NO lock held) — run the factory. The wrapper ctor's RegisterHandle sees our reservation (same thread + handle) and suppresses publication, so a half-built wrapper is never visible to other threads.
3. Phase 3 (under shard lock, O(1)) — publish the finished wrapper into instances, drop the reservation, then signal the gate so waiters wake to a fully-constructed object.

Because no shard lock is ever held across the factory, a thread holds at most one shard lock at any instant → the cross-shard cycle is impossible, not merely unlikely. In-flight handles live in a separate per-shard reservations map, so instances still only ever holds finished wrappers and GetInstance / DeregisterHandle / the diagnostics aggregators needed no changes.

This also avoids the rejected "construct then dispose the duplicate" dedupe strategies: exactly one wrapper is ever constructed per handle, so there is no loser to dispose and no observable half-built object that user code might already hold.

Residual hazards (documented, tested where deterministic):

• Re-entrant same-thread, same-handle construction (a factory re-enters GetOrAddObject for the same handle) is detected and thrown — never hung.
• A genuine cross-object construction-time dependency cycle between two threads is an application bug, far rarer than the structural hazard removed here (not turned into a hanging test by design).
• Per-construction ManualResetEventSlim allocation on the contended path; GC reclaims it.

Test status

• Builds clean (Debug net10.0).
• Object-lifecycle / concurrency suite passes with active sharding; cross-shard aggregation in GarbageCleanupFixture works (no leak/exception drain).
• New SKHandleDictionaryReservationTest (white-box, drives the registry with a non-owning FakeNativeObject) pins every claim of the reservation-gate design — all green:
  • ConcurrentSameHandleConstructsExactlyOnce — 32 threads + barrier; factory runs once; all Same.
  • WaitersOnlyObserveFullyConstructedWrapper — publication safety; waiters always observe a fully-built wrapper.
  • ReentrantSameThreadSameHandleThrows — fail-fast under a timeout guard.
  • FactoryFailureClearsReservationAndAllowsReconstruction + ConcurrentFactoryFailureRecoveredByWaiter — a throwing owner clears the reservation; a waiter reconstructs.
  • NestedDifferentHandleConstructionRegistersBoth — single-thread nesting completes.
  • CrossShardOppositeOrderConstructionDoesNotDeadlock — the live deadlock regression: probes two distinct shards via GetLockFor identity (skips if ShardCount == 1), drives the exact AB-BA interleaving, asserts completion under a 15s timeout. The old factory-under-lock design hangs here.
  • DisposeProtectedFreshConstructionSetsFlag — Phase 3 sets IgnorePublicDispose under the write lock.
• The only run-aborting failure in the broader suite is the pre-existing SKManagedStream.OnReadManagedStream NRE host-crash — intermittent, reproduces identically on the Rework the lifecycle of singleton instances #4080 base branch, unrelated to striping.

---

cc @ramezgerges — purely exploratory, building on your #4080 work. Not asking you to take this on; just parking the data + the deadlock-free redesign so we can decide together whether it's worth a real PR later.

[github-actions]

📦 Try the packages from this PR

Warning

Do not run these scripts without first reviewing the code in this PR.

Step 1 — Download the packages

bash / macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4102

PowerShell / Windows:

iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4102"

Step 2 — Add the local NuGet source

dotnet nuget add source ~/.skiasharp/hives/pr-4102/packages --name skiasharp-pr-4102

More options

Option	Description
--successful-only / -SuccessfulOnly	Only use successful builds
--force / -Force	Overwrite previously downloaded packages
--list / -List	List available artifacts without downloading
--build-id ID / -BuildId ID	Download from a specific build

Or download manually from Azure Pipelines — look for the nuget artifact on the build for this PR.

Remove the source when you're done:

dotnet nuget remove source skiasharp-pr-4102

[mattleibow]

Update: reservation-gate two-phase construction

Pushed 28c63c1 reworking the striped registry so the object factory never runs under a shard lock. This removes the cross-shard deadlock hazard the first striping prototype introduced, while keeping exactly-once construction.

The hazard it removes

The initial prototype ran the factory while holding the shard lock. Nested, different-handle construction could then form a classic AB-BA cross-shard cycle:

• thread 1 holds shard A, its factory constructs a child on shard B → wants B
• thread 2 holds shard B, its factory constructs a child on shard A → wants A

With a single global lock this could never happen (one lock, no ordering). Striping reintroduced it.

How it works

GetOrAddObject is now three phases:

1. Phase 1 (under shard lock, O(1)) — dedup against a live instance; else if another thread is already building this handle, wait on its per-handle gate and retry; else install our own Reservation and release the lock.
2. Phase 2 (NO lock held) — run the factory. The wrapper ctor's RegisterHandle sees our reservation (same thread + handle) and suppresses publication, so a half-built wrapper is never visible to other threads.
3. Phase 3 (under shard lock, O(1)) — publish the finished wrapper into instances, drop the reservation, then Set() the gate so waiters wake to a fully-constructed object.

Because no shard lock is ever held across the factory, a thread holds at most one shard lock at any instant, so the cross-shard cycle is structurally impossible — not merely unlikely.

In-flight handles live in a separate per-shard reservations map, so instances still only ever holds fully-constructed wrappers and GetInstance / DeregisterHandle / the diagnostics aggregators needed no changes.

What it solves vs. the "construct then dispose the loser" approaches

Earlier we worried about dedupe strategies that build a C# object and then dispose the duplicate — unacceptable because user code may already hold (and act on) the losing handle/wrapper. The reservation gate sidesteps that entirely: only one wrapper is ever constructed per handle, so there is no loser to dispose and no observable half-built object.

Residual hazards (documented, tested where deterministic)

• Re-entrant same-thread, same-handle construction (a factory re-enters GetOrAddObject for the same handle): detected and thrown — never hangs. Test: ReentrantSameThreadSameHandleThrows.
• Genuine cross-object construction-time dependency cycle between two threads: an application bug, far rarer than the structural hazard removed here. Not turned into a hanging test by design.
• Per-construction ManualResetEventSlim allocation: cheap, only on the contended/in-flight path; GC reclaims it.

Tests added (SKHandleDictionaryReservationTest)

A white-box FakeNativeObject : SKObject (owns: false, no native memory) drives the registry directly:

• ConcurrentSameHandleConstructsExactlyOnce — 32 threads + barrier; factory runs once; all Same.
• WaitersOnlyObserveFullyConstructedWrapper — publication safety; a ctor delay between base-ctor and an initialized marker; every waiter observes FullyConstructed == 1.
• ReentrantSameThreadSameHandleThrows — fail-fast, with a timeout guard.
• FactoryFailureClearsReservationAndAllowsReconstruction + ConcurrentFactoryFailureRecoveredByWaiter — a throwing owner clears the reservation; a waiter reconstructs.
• NestedDifferentHandleConstructionRegistersBoth — single-thread nesting completes.
• CrossShardOppositeOrderConstructionDoesNotDeadlock — the live deadlock regression: probes two distinct shards via GetLockFor identity (skips if ShardCount == 1), drives the exact AB-BA interleaving, asserts completion under a 15s timeout. The old factory-under-lock design would hang here.
• DisposeProtectedFreshConstructionSetsFlag — Phase 3 sets IgnorePublicDispose under the write lock.

All green locally (net10.0, Debug → THROW_OBJECT_EXCEPTIONS on). This stays an experiment for benchmarking; the goal is to learn whether striping carries its weight before proposing anything for the core.