fix(wasm): stop a poisoned MSBuild node baking an empty bundle in silence - #657
Merged
Conversation
…ipped The bake stages scoped CSS/JS into obj/.../rask-scoped and registers them as computed static web assets in the build pass, trusting them to flow into the publish manifest. When that link breaks -- most reliably by building one project in both WasmBuildNative modes through a single obj/ -- the published bundle has no /_rask/a/ at all and nothing says so. The app builds, publishes, boots and renders; only its scoped CSS/JS is missing, every scoped URL 404s, and for an app whose scoped JS owns something load-bearing it presents as a hung page. That cost two people a debugging session each (#650, #652). The publish now compares what was staged against what shipped. Both halves are load-bearing: checking only the published side would fail every project that legitimately has no scoped assets, and checking only staging would never fire. The error names the cause and the fix. BakeScopedAssetsTask.FailOnEmpty could not have caught this, and despite its own documentation claiming it is 'wired to true on the dotnet run hook', nothing has ever set it -- the single call site leaves it at its default. It fires only when the bake RUNS and writes zero, whereas here the bake runs fine and the break is downstream. Its docs now say both things. Verified against a real publish in all three states: staged-but-not-published errors with the message, neither passes, both passes. Closes #650
…tive publish Reported and measured on another machine: clearing only the intermediates still produced a publish with zero scoped assets, and adding bin/Release/net10.0-browser to the clean fixed it. The reported mechanism is that a stale no-native bin lets the publish treat the compile as up to date, so the bake never re-runs and the just-deleted staging is never rebuilt. That mechanism is NOT confirmed, and the comment says so: the same sequence reproduces cleanly here in three variations, including the full solution build, each time re-running the bake and shipping 6 files. The deciding variable is still unknown. Taking the superset clean anyway: the cost is one recompile of a project whose native relink dominates this step regardless, and the failure it guards against is silent and has now cost several sessions. The publish-time verification added in this branch turns the same failure into a build error either way, so this is belt as well as braces. Also worth recording: dotnet publish does not clean its output directory, so an earlier good publish can leave _rask in place and make a broken publish look fine. That confound probably invalidated the twice-back-to-back verification in #652.
The staged-vs-published comparison has a hole, and it is the one that matters: when the bake does not run, the staging dir is absent, so 'this project has no scoped assets' and 'the bake was skipped' are the same observation and the guard stays silent through exactly the failure it was written for. Found by reproducing #650 live -- the E2E gate failed on this branch after a rebase, and the broken publish had neither wwwroot/_rask NOR any staging. Since the bake target unconditionally RemoveDir+MakeDirs its staging, a missing staging dir is proof it never ran, which also confirms the mechanism the other session inferred and I could not reproduce. The bake now records that it ran, and a publish that never baked and shipped no scoped assets fails with a message pointing at the two conditions that skip the target. Both halves of that condition are load-bearing: an incremental publish can legitimately skip the build pass while correct assets already sit in wwwroot. Verified: fires on the live broken publish; silent for a normal playground publish (6 files); silent for Rask.Example.Wasm.Jobs, which legitimately bakes zero.
The guards cover two of the three shapes this failure takes. The third -- a bake that RUNS and produces zero for an app that does have scoped assets -- is not covered and cannot be, from outside the bake: an empty staging dir is indistinguishable from a project that legitimately has none. Reproduced live on this branch in both covered and uncovered shapes, which is why the claim is narrowed rather than left as 'closes'. FailOnEmpty cannot be wired to cover it: registryResolved is true whenever Rask.Core merely loads, so enabling it would falsely fail every WASM app with no scoped assets, Rask.Example.Wasm.Jobs included. #650 stays open.
Two corrections to claims I made in this branch, both disproven by evidence since: The bake is not being skipped. I read a missing rask-scoped/_rask/a as 'the bake never ran', but that path only exists when files are written -- the staging dir itself was present. Both my reproductions were the same shape another session found in its build log: after the no-native solution build, the native publish's bake RUNS and writes ZERO files for an app that plainly has scoped assets. The bin clean is not the fix. It was added on an A/B that looked decisive and has since been contradicted by reproductions on branches that already had it. It stays because it is cheap, and the comment now says plainly that it does not fix anything. So neither guard in this branch covers #650, which stays open: nothing outside the bake can distinguish 'baked zero because something went wrong' from 'baked zero because there is nothing to bake'. The guards still cover two real shapes and turn those into named build errors, which is worth having on its own terms.
pal-tamas
force-pushed
the
worktree-bake-fail-on-empty
branch
from
August 8, 2026 10:05
8d9185d to
c4e00a1
Compare
…ence Root cause, from another session's binlog: BakeScopedAssetsTask inspects the built assemblies with Assembly.LoadFrom, and MSBuild reuses worker nodes. A publish landing on a node that already loaded an assembly of the same simple name throws FileLoadException; the task caught it, logged at MessageImportance.Low and continued, so it baked nothing and reported success. The published app then 404s on every scoped CSS/JS URL, which reads as a broken app rather than a broken build. It is node-STATE dependent, not a race: once a reused node has loaded that assembly every publish on it fails, and with a cold pool every publish passes. That is why it looked intermittent and why every clean either of us tried appeared to work exactly once -- we were sampling whether the pool happened to be warm. Measured here: 3 failures in 4 consecutive publishes on a warm pool, 0 in 4 with -nodeReuse:false; another session measured 6/6 versus 0/6. Two changes, neither of them the real fix: - The gate publishes the playground with -nodeReuse:false. Measured, and it only protects that one call site. - The task now fails when it baked zero AND skipped an assembly because one of that name was already loaded. That combination is never a legitimate empty bake. Conditioned on both so a bake that still produced its files cannot be broken by it. Honesty about the second one: it is written from the binlog, and I have NOT seen it fire. I could not reproduce the poisoned-node state on demand -- and node reuse pins the task assembly too, so a reused node keeps running the OLD task and hides any change to it, which is its own small trap for anyone testing this. The real fix is a fresh AssemblyLoadContext per invocation, so a reused node cannot collide at all. That is not in here: Rask.Wasm.Tasks targets netstandard2.0 where AssemblyLoadContext is unavailable at compile time, so it needs multi-targeting and a runtime-conditional UsingTask, with packaging consequences that deserve their own PR. #650 stays open for it.
pal-tamas
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #650, which stays open for the real fix. The root cause is now known, and this makes the failure impossible to ship silently — but the cure belongs in a separate PR with packaging consequences.
Root cause
BakeScopedAssetsTaskinspects the built assemblies withAssembly.LoadFrom, and MSBuild reuses worker nodes. A publish that lands on a node which already loaded an assembly of the same simple name (from an earlier build in the same session) throwsFileLoadException; the task caught it, logged atMessageImportance.Lowandcontinued — so it baked nothing and reported success. The published app then 404s on every scoped CSS/JS URL, which reads as a broken app rather than a broken build.It is node-state dependent, not a race: once a reused node has loaded that assembly, every publish on it fails; with a cold pool, every publish passes. That single fact explains every dead end this took — "fresh worktree", "stale
obj", "stalebin", "no-native build first". Each of them appeared to work exactly once, because we were all sampling whether the pool happened to be warm.Measured here — identical inputs, same clean between each:
The session on #650 measured 6/6 versus 0/6, and captured the binlog naming the
FileLoadException.What ships here
-nodeReuse:falseon the gate's playground publish. Measured, and honest about its scope: it protects that one call site. Anyone publishing a Rask WASM project twice in a warm MSBuild session — ordinary IDE development — still hits it._RaskVerifyPublishedScopedAssets) for the adjacent shapes: baked-but-not-shipped, and never-baked-and-nothing-shipped. Verified firing against real publishes, withsamples/Rask.Example.Wasm.Jobs— which legitimately bakes zero — staying quiet.FailOnEmptydocumentation corrected. It claimed to be "wired totrueon thedotnet runhook"; nothing has ever set it. That claim is what led the other session to conclude the guard had failed to fire when there was no guard.What is NOT here
The real fix: a fresh
AssemblyLoadContextper invocation, so a reused node cannot collide at all.Rask.Wasm.Taskstargetsnetstandard2.0, whereAssemblyLoadContextis unavailable at compile time, so it needs multi-targeting plus a runtime-conditionalUsingTask— packaging consequences that deserve their own PR. The other session is taking it; I'll review the packaging side.(
MetadataLoadContextis not the answer, for the record: the task doesn't merely read metadata, it invokesRefreshAll/GetByHashon the loaded assemblies.)Honesty about verification
I have not seen guard #2 fire. It is written from the binlog. I could not reproduce the poisoned-node state on demand, and there is a trap worth knowing for whoever tests the ALC change: node reuse pins the task assembly too, so a reused node keeps running the previous
Rask.Wasm.Tasks.dlland hides your edit —dotnet build-server shutdownfirst, or you are measuring the old task.Both
objandbincleans are now described as cheap insurance rather than fixes. Both comments previously claimed otherwise, in this branch and onmain.Testing
dotnet format+ warnings-as-errors + full unit suite green (320);Rask.Wasm.Tasks.Tests14 green.-nodeReuse:falsemeasured 4/4 clean against 1/4 baseline.