Run test262's staging/ directory too

[lahma]

Draft — blocked on Test262Harness.Console 1.1.2. The tool version this pins does not exist on nuget.org yet; it needs lahma/test262-harness-dotnet#173 released. Everything below was measured against that branch packed locally as 1.1.2. It is also merge-order dependent — see the bottom.

Why staging/ was missing, and the one-line reason

We generate annexB, built-ins, intl402 and language. staging/ has never been generated, and ExcludedDirectories is empty — so nobody ever decided to leave it out. The exclusion was structural: Test262Harness.Console's settings file simply had no way to name the set.

Three of the four pieces were already in the tool:

• Test262StreamOptions.SubDirectories existed, defaulting to those four.
• Test262Stream.EnumerateTestFiles already recursed (SearchOption.AllDirectories), so a deeper tree needs nothing new.
• TestSuiteGenerator.Generate already iterated stream.Options.SubDirectories.

What was missing was the binding: TestSuiteGeneratorOptions — which is what Test262Harness.settings.json deserializes into — had no SubDirectories property, and GenerateCommand's configureOptions lambda set only LogInfo/LogError, so the stream kept its built-in default whatever the settings file said. lahma/test262-harness-dotnet#173 adds the property and assigns it; this PR bumps to the 1.1.2 that carries it and opts in with one array entry.

What it buys: both defects fixed in 4.16.0 have their only upstream test there

• staging/sm/Proxy/global-receiver.js — installs a Proxy as the global object's [[Prototype]] and asserts the get/set traps receive the global itself as the receiver. That is the defect fixed in Ask an exotic global prototype for a bare identifier through [[Get]] #2946.
• staging/sm/global/parseInt-01.js — asserts parseInt("-0x10", 16) === -16 (line 117) and that parseInt("-0", 10) is -0 (line 60). Both were fixed in Strip the sign before testing parseInt's argument for a hex prefix #2945, and neither assertion is reachable anywhere else in the suite: all 55 files under built-ins/parseInt/ use unsigned hex.

Both pass, in both modes:

Passed Sm_Proxy("staging/sm/Proxy/global-receiver.js",False) [157 ms]
Passed Sm_Proxy("staging/sm/Proxy/global-receiver.js",True)  [157 ms]
Passed Sm_global("staging/sm/global/parseInt-01.js",False)   [173 ms]
Passed Sm_global("staging/sm/global/parseInt-01.js",True)    [173 ms]

Cost

	before	after
generated test cases	99,901	102,664
passed	99,779	102,322
skipped (excluded)	122	345
failed	0	0
dotnet test duration	2 m 48 s	3 m 00 s
wall clock including build	191 s	199 s

staging/ is 1,483 non-fixture files — 1,409 of them under sm/ (SpiderMonkey's own suite, contributed upstream), the rest explicit-resource-management (53), built-ins (8), top-level-await (4), source-phase-imports (4), decorators (3), set-methods (3), Temporal (2), Uint8Array (1), plus 4 at the top level. 131 are excluded, 1,352 pass — a 91% pass rate for a directory that has never been run.

The harness-file change

Test262Stream.GetHarnessFiles() enumerates only the top level of harness/, and an includes: entry is looked up by the exact string the test's frontmatter wrote. The SpiderMonkey ports write includes: [sm/non262-Set-shell.js, ...], and those 11 helpers live in harness/sm/. So TestHarness.cs now enumerates the whole tree off State.Test262Stream.Options.FileSystem and keys State.Sources on the path relative to harness/.

A top-level file's relative path is just its file name, so every pre-existing include resolves byte-identically — and the run proves that rather than merely asserting it: all 99,901 pre-existing cases still pass, including the 32 top-level harness scripts every one of them loads.

Exclusions

All 131 are under staging/, grouped under a // === STAGING EXCLUSIONS === banner with a prose reason per group, matching the file's existing intl402 convention. Every entry is commented with what removes it.

Blocked on a sibling PR (2) — see merge order below

file	reason
staging/Temporal/removed-methods.js	Five Temporal members the proposal dropped are still present — removed by the fix for #3010
staging/sm/String/replace-math.js	An over-long replace result must raise RangeError; today the length overflows to int.MinValue and surfaces as ArgumentOutOfRangeException out of the BCL — removed by the fix for #3011

Hangs, or exhausts the machine (20)

A test that hangs or OOMs is worse than one that fails, so these are excluded rather than left red. staging/sm/extensions/recursion.js is the sharpest: it recurses without bound, and since Jint's default MaxRecursionDepth is -1 the CLR stack overflows and takes the entire test host with it, every other test in the process included.

staging/sm/extensions/recursion.js, staging/sm/Array/to-length.js ({length: Infinity} array generics), the eight staging/sm/Date/dst-offset-caching-*-of-8.js (which additionally drive multi-GB allocation), staging/sm/generators/delegating-yield-9.js, staging/sm/regress/regress-610026.js, staging/sm/regress/regress-619003-1.js — and, found by running the suite, staging/sm/Function/has-instance-jitted.js, staging/sm/JSON/parse-mega-huge-array.js, staging/sm/RegExp/unicode-braced.js, staging/sm/RegExp/unicode-class-braced.js, staging/sm/RegExp/unicode-ignoreCase.js, staging/sm/expressions/nullish-coalescing.js, staging/sm/regress/regress-1507322-deep-weakmap.js. Each burns the whole 30-second budget, twice over (both modes), and together they starved the run badly enough to time out unrelated tests elsewhere in the suite.

Conformance gaps the stable directories never reach (109)

group	n	reason
Cross-realm error identity	13	$262.createRealm() works, but an error raised while running another realm's function is constructed from the current realm's intrinsics, so e instanceof otherGlobal.TypeError is false. The created realm's global also carries no $262 of its own, so otherGlobal.$262.detachArrayBuffer is undefined
Legacy Function.prototype.caller / arguments.callee.caller	12	The Annex B accessors' sloppy-mode observable behaviour — the calling function, null at the top of the stack, poisoned on strict functions, transparent through eval frames — is not reproduced
Argument validation on a bad this or index	10	No error raised, or the wrong one. Includes SpiderMonkey's own toSource extension, which Jint does not implement at all — that half is removed by the test moving, not by Jint changing
Set-method operation order	7	GetSetRecord reads size, then has, then keys, and coerces size exactly once; Jint's order and coercion count differ
Iterator protocol / IteratorClose	7	Not invoked, or invoked at the wrong point, on the abrupt paths of Array.from, the Map/Set constructors, for-of and array destructuring
RegExp lastIndex and case-insensitive matching	7	The Symbol.match / Symbol.replace / matchAll lastIndex bookkeeping (including -0 and the per-iteration ToLength recomputation), and matching across the Latin-1 boundary
Parser early errors	7	Not raised by Acornima, or raised as the wrong type: invalid parameter list, super() inside an arrow inside a direct eval in a derived constructor, a duplicate destructuring binding, a strict-mode legacy octal literal, yield as an identifier, a field initializer's TDZ
Math precision	6	acosh/asinh/atanh/cbrt/expm1/log1p delegate to the BCL, which differs in the last few ULPs and flushes 1e-300 to zero
Date parsing and clipping	5	The implementation-defined fallback parser accepts and rejects different strings, two-digit years map differently, TimeClip does not reject every out-of-range value, and Symbol.toPrimitive throws a non-object
Annex B B.3.3 block-scoped function hoisting	5	The var-scoped alias a sloppy-mode block function creates is not synthesised the way the spec's FunctionDeclarationInstantiation amendment describes
Function source text and inferred names	5	Function.prototype.toString does not return the original source for class expressions and some method forms, and NamedEvaluation does not reach every position the spec names
Array exotic [[DefineOwnProperty]] and generics ordering	5	Truncating length past a non-configurable sparse element stops at the wrong index, sort/unshift observe the wrong element sequence, and the strict-mode generics do not report a failed write
Number conversion and formatting	4	toFixed/toPrecision lose the exact mathematical value for extreme magnitudes and subnormals, and ToPrimitive on a Number wrapper consults the wrong method
Proxy trap invariants and trap sequence	4	The exact trap order the array built-ins are specified to perform, plus a defineProperty invariant check of Jint's own that fires where it should not
Array.from applied to an arbitrary constructor	2	The construct-vs-ArrayCreate branch, and CreateDataPropertyOrThrow on a read-only element or a non-extensible target
Mapped arguments	2	The exotic object's link to the formal parameters, and what an object created from it inherits
Source phase imports (stage 3)	2	Supported for WebAssembly-shaped sources only, so import source of a JavaScript module is refused outright and the AbstractModuleSource prototype chain differs
String case mapping	2	Uses the BCL's culture-aware tables rather than the Unicode Default Case Conversion the spec requires, so dotless i and some supplementary code points map differently
One-offs	4	Iterator.from's next() result validation, the Map constructor's per-entry type check, Symbol.keyFor over a cross-realm registry, and a module namespace re-exported under a string name

One pre-existing fragility this exposed

Four language/literals/regexp/S7.8.5_*_T2.js tests join the existing _slowTestFiles list in Test262Test.cs. They loop 0x0000..0xFFFF calling eval() on a fresh regexp literal per code unit, so every iteration is a parse. The whole Literals_regexp group still runs in 6 seconds on its own; these four only exceed the 30-second wall clock once staging adds ~2,800 cases to the parallel run. That is CPU starvation, not the engine getting slower — which is exactly what that list's comment already anticipates for the decodeURI byte-range tests sitting next to them.

Merge order

This PR must merge after both of:

• the PR for Temporal: five members removed from the proposal are still present #3010 (Temporal removed methods) — otherwise staging/Temporal/removed-methods.js is red
• the PR for A string past the implementation limit overflows the builder instead of throwing RangeError #3011 (String.prototype.replace length overflow) — otherwise staging/sm/String/replace-math.js is red

Both are excluded here with the issue number named in the comment, so whichever lands first, the follow-up is a two-line deletion. And the whole thing stays a draft until Test262Harness.Console 1.1.2 is on nuget.org (lahma/test262-harness-dotnet#173).

Closes #3008

🤖 Generated with Claude Code