Skip to content

Run the repository's own host tests under Release-mode contract verification - #2866

Merged
lahma merged 1 commit into
sebastienros:mainfrom
lahma:test/release-verification-leg
Jul 29, 2026
Merged

Run the repository's own host tests under Release-mode contract verification#2866
lahma merged 1 commit into
sebastienros:mainfrom
lahma:test/release-verification-leg

Conversation

@lahma

@lahma lahma commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

#2864 made the host-contract verifiers reachable from the shipped Release package, and its AssemblyLoadContext test proves the Jint.EnableHostContractVerification switch turns them on. But nothing exercised the full host-test surface that way — the exact configuration an embedder is told to use. CI gates in Release, where the verifiers are dormant; broadly they ran only in Debug, on a developer's machine. A verifier that misbehaves under the Release JIT (tiering, inlining, dead-branch folding), or a gating bug, would have shipped unseen. This dogfoods the documented recipe in-repo.

This PR touches only test projects, the PR workflow and AGENTS.md. No shipped code changes.

The opt-in

JINT_HOST_CONTRACT_VERIFICATION=1 (or true), turned into the AppContext switch by a [ModuleInitializer] in Jint.Tests/HostContractVerificationSwitch.cs, compiled into both Jint.Tests and Jint.Tests.PublicInterface:

JINT_HOST_CONTRACT_VERIFICATION=1 dotnet test -c Release Jint.Tests/Jint.Tests.csproj
JINT_HOST_CONTRACT_VERIFICATION=1 dotnet test -c Release Jint.Tests.PublicInterface/Jint.Tests.PublicInterface.csproj

A module initializer is the only hook early enough. The gate is a static readonly bool read once at type initialization, so a fixture, a collection or a class constructor all run after some Jint type has already been touched. It also needs nothing per target framework: net472 has no ModuleInitializerAttribute in its BCL, but the repository's global PolySharp reference emits one and the compiler matches the attribute by name rather than by identity. Verified on net472 specifically — it is the TFM most likely to break this, and it does not.

A dedicated leg, not a blanket flip

The Release probe-count and no-descriptor pins are the regression net for the claim that the disabled gate folds to zero cost, and they can only mean that in a run where nobody asked for verification. So the default Release run stays switch-off.

Every expectation that differs between the two configurations now selects at runtime from HostContractVerificationSwitch.Enabled instead of on #if DEBUG:

File What moved
HostObjectProbeCountTests the #if DEBUG constants became a per-kind ProbeCosts table — [InlineData] arguments must be compile-time constants, so the counts moved out of the attributes into the method body
HostObjectSemanticsTests eight const int → static readonly int
HostObjectProbeOverrideTests the descriptor-materialization assertions, plus 3 + 2 configuration-specific tests
HostArrayLikeHasIndexTests the TryGetIndex-not-called assertion, plus 2 + 2 configuration-specific tests
ObjectConverterRegistrationTests the ShouldRead helper, plus 3 + 1 configuration-specific tests
Jint.Tests/Runtime/ArrayLikeObjectLaneTests the HasIndex/TryGetIndex agreement count
Jint.Tests/Runtime/SharedObjectShapeTests the two builtin-shape no-materialization assertions

The nine tests that only make sense with the verifiers on, and the five that only make sense with them off, are now compiled in both configurations and skipped with SkipUnless, so a run states which configuration it is rather than silently containing fewer tests.

Measured, not assumed: every Release+switch count equals the Debug one. The extra probes really are only the verifier's own, and nothing else about the two builds differs on these paths.

Ordering, pinned from both sides

  • Jint.Tests.Runtime.HostContractVerificationSwitchTests checks the harness flag against Jint's own gate — Jint.Tests has InternalsVisibleTo, so it can read HostContractVerification.Enabled directly. The two are computed from different inputs (the switch vs. the environment variable) and can only agree if the module initializer really did run before the first use of any Jint type.
  • HostContractVerificationTests.TheHarnessSetsTheSwitchEarlyEnoughForJintToObserveIt proves the same thing from behaviour alone, which is all Jint.Tests.PublicInterface may use: a host whose ProbeOwnProperty contradicts its own GetOwnProperty throws exactly when the verifiers are running.
  • TheDefaultIsOnForADebugBuildAndOffForTheShippedRelease now asserts off unless the harness asked, so the zero-cost default stays pinned in the default run and the opt-in is pinned in the verification leg.

CI

One new job in pr.yml, linux - host-contract verification, running both suites in Release with the variable set. One OS is enough — the verifiers are platform-independent and the switch is set the same way everywhere — and it deliberately does not multiply the four-OS matrix.

Gate

All four legs green locally, net10.0 and net472 in each:

Leg Jint.Tests (net10.0 / net472) Jint.Tests.PublicInterface (net10.0 / net472)
Release, switch off (default) 4622 / 4541 passed, 0 failed 1240 / 1239 passed, 9 skipped, 0 failed
Release, switch on (new leg) 4622 / 4541 passed, 0 failed 1244 / 1243 passed, 5 skipped, 0 failed
Debug 4622 / 4541 passed, 0 failed 1243 / 1243 passed, 5 skipped, 0 failed

dotnet build -c Release (whole solution): 0 errors, 0 compiler warnings. The one remaining warning is a pre-existing MSBuild MSB3277 (System.Memory 4.0.1.2 vs 4.0.2.0) in Jint.Tests.CommonScripts on net472, a project this PR does not touch. Test262 was not run — no shipped code changed.

Red-first: with the mechanism in and the tests untouched, the switch-on Release run failed 41 tests on net10.0 and 45 on net472, which is both the proof that the switch reached Jint on both TFMs and the list of expectations that needed adapting.

@lahma @claude
Run the repository's own host tests under Release-mode contract verif…
906536f 
…ication

sebastienros#2864 made the host-contract verifiers reachable from the shipped Release
package, and its AssemblyLoadContext test proves the AppContext switch turns
them on. Nothing exercised the full host-test surface that way, though: CI gates
in Release, where the verifiers are dormant, and they ran broadly only in Debug
on a developer's machine. A verifier that misbehaves under the Release JIT —
tiering, inlining, dead-branch folding — or a gating bug would have shipped
unseen. This runs the documented embedder recipe in-repo.

The opt-in is JINT_HOST_CONTRACT_VERIFICATION=1, turned into the AppContext
switch by a [ModuleInitializer] in Jint.Tests/HostContractVerificationSwitch.cs,
compiled into both Jint.Tests and Jint.Tests.PublicInterface. A module
initializer is the only hook early enough: the gate is a static readonly bool
read once at type initialization, so a fixture, a collection or a class
constructor all run after some Jint type has been touched. It needs nothing per
target framework either — net472 has no ModuleInitializerAttribute in its BCL,
but the repository's global PolySharp reference emits one and the compiler
matches the attribute by name.

It is a dedicated leg, not a blanket flip. The Release probe-count and
no-descriptor pins are the regression net for the claim that the disabled gate
folds to zero cost, and they can only mean that in a run where nobody asked for
verification. So the default Release run stays switch-off, and every expectation
that differs between the two configurations now selects at runtime from
HostContractVerificationSwitch.Enabled instead of on #if DEBUG:
HostObjectProbeCountTests (whose [InlineData] counts became a per-kind cost
table, since an attribute argument cannot be runtime-selected),
HostObjectSemanticsTests, HostObjectProbeOverrideTests,
HostArrayLikeHasIndexTests, ObjectConverterRegistrationTests,
ArrayLikeObjectLaneTests and SharedObjectShapeTests. The nine tests that only
make sense in one configuration are now compiled in both and skipped with
SkipUnless, so a run states which configuration it is rather than silently
containing fewer tests. Measured rather than assumed: every Release+switch count
equals the Debug one — the extra probes really are only the verifier's own.

Two tests pin the ordering the mechanism depends on, from both sides:
HostContractVerificationSwitchTests checks the harness flag against Jint's own
gate (Jint.Tests has InternalsVisibleTo), and
HostContractVerificationTests.TheHarnessSetsTheSwitchEarlyEnoughForJintToObserveIt
proves it from behaviour alone — a host whose ProbeOwnProperty contradicts its
GetOwnProperty throws exactly when the verifiers are running.

CI gains one job, `linux - host-contract verification`, running both suites in
Release with the variable set. One OS is enough: the verifiers are
platform-independent and the switch is set the same way everywhere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uV6H9cTntzsoKiaJRBn4f
@lahma
lahma enabled auto-merge (squash) July 29, 2026 11:01
@lahma
lahma merged commit 01ed368 into sebastienros:main Jul 29, 2026
5 checks passed
@lahma
lahma deleted the test/release-verification-leg branch July 29, 2026 11:08
This was referenced Aug 14, 2026
Closed
Merged
Closed
Closed
Closed
Merged
Closed
Merged
legrab added a commit to legrab/pocok that referenced this pull request Aug 18, 2026
@legrab
Bump Jint from 4.15.3 to 4.16.0 (#56)
ef20afa 
Updated [Jint](https://github.com/sebastienros/jint) from 4.15.3 to
4.16.0.

<details>
<summary>Release notes</summary>

_Sourced from [Jint's
releases](https://github.com/sebastienros/jint/releases)._

## 4.16.0

Jint 4.16.0 is a **correctness- and reliability-focused release**:
alongside asynchronous module loading, proper tail calls and four new
iterator built-ins, a pre-tag review swept the whole engine and fixed
what it found — including long-standing defects that predate this cycle.
**No option defaults changed.** Behaviour changes to note up front:
`JSON.stringify` and other machine-readable output now format
invariantly under every host culture — under Swedish or Finnish locales
on .NET 8+ it used to emit a Unicode minus sign no JSON parser accepts;
`JSON.parse` now rejects trailing commas as the grammar requires; bare
identifiers at global scope resolve through the global's prototype chain
per spec; `IModuleLoader.Resolve` is consulted once per (referrer,
specifier) pair, so a loader using it as a per-import access-control
checkpoint should move the check to `LoadModule`; and an inconsistent
sort comparator now finishes with an implementation-defined order on
every target framework instead of hanging (net462/netstandard) or
throwing a CLR exception at script (net8+).

### Highlights

**Proper tail calls (#​2975).** Strict-mode calls in tail position reuse
their frame, so `"use strict"` tail recursion runs in constant stack —
the first ES2015 PTC implementation among the .NET engines.

**Asynchronous module loading (#​2872).** `IAsyncModuleLoader` and the
`AsyncModuleLoader` template let a host fetch module source over I/O
without blocking a thread; `Engine.Modules.StartImport` returns an
operation a game loop drives via `ProcessTasks()`, and `ImportAsync`
awaits without holding a thread. The spec's load phase now exists as
written, a warm-cache async loader keeps the blocking `Import` fully
synchronous, and the blocking drain wakes on a work-arrived signal
instead of polling. A module served over a transport keeps its whole url
as `Module.Location` so its own relative imports resolve, a deferred
namespace evaluates its module instead of exposing uninitialized
bindings, and an import abandoned by a global snapshot restore reports
itself faulted instead of polling forever.

**The process no longer dies for recoverable reasons.**
`Options.LimitRecursion` used to kill the host process for most useful
limits — the constraint fired, and the unwind itself overflowed the
stack; exception filters now let it unwind ~7× deeper. The new opt-in
`Options.Constraints.StackOverflowGuard` converts unbounded recursion —
reachable through eighteen distinct routes, `new`, accessors, coercions
and Proxy traps included — from a process kill into a catchable
`RangeError`, exempting strict tail calls, which grow no stack. And a
family of CLR exceptions that escaped `engine.Evaluate` past every
script `catch` are now proper JavaScript errors or correct results:
sorting with an inconsistent comparator, destructuring with a
function-valued default (`const { onChange = () => {} } = opts`),
`toLocaleString` outside `DateTime`'s range, typed-array
`defineProperty` without a value, `DataView` reads at 2³¹,
`String.replace` `$'` with a lying exec, and the first instant of year
10000.

**New built-ins.** `Iterator.prototype.join`, `chunks`, `windows` and
`includes`; `take`/`drop` now throw `RangeError` for a finite limit
above 2^53−1 per the updated proposals.
`Intl.Locale.prototype.getCollations` reports CLDR-cited collation data
that `Intl.Collator` accepts in full, a malformed `collation` option is
a `RangeError`, and `Intl.supportedValuesOf("collation")` derives from
the same lists so the three can never drift.

**Conformance, from a review that ran what the suite does not.** Two of
the fixed defects had test262 coverage only under the never-generated
`staging/` directory, and several had none at all: `parseInt` strips the
sign before testing for a hex prefix, so `parseInt("-0x10")` is −16; a
suspended `finally` no longer swallows a pending `break`/`continue`; a
Proxy (or exotic host object) as the global's prototype answers bare
identifiers through its `get` trap; `Date.prototype.toISOString` emits
the spec's six-digit expanded year and round-trips through `Date.parse`
in every spelling including year 0; iterator helpers close their
receiver exactly once and only when the spec says so, and carry their
own `@@​toStringTag`; `Map`/`Set` `size` is the prototype accessor the
spec defines rather than a phantom own property; a Proxy's
`defineProperty` trap receives the partial descriptor the caller wrote;
a string's `@@​iterator` is read once, with the primitive as receiver;
`Array.prototype.join` re-asks the array when a side effect fills a hole
mid-join; a direct eval reaches the enclosing function's `arguments` in
both modes; and `Temporal.Now` drops the methods the proposal removed.

**Embedder surface.** `OperationDeadlineConstraint` bounds a whole
multi-entry host operation; `ScriptPreparationOptions.StaticAnalysis`
trades prepare-time analysis for per-engine materialization on shared
graphs; `ModuleFactory.LocationOf` exposes the module-naming rule a host
must match; `Engine.Advanced.HostDefined` carries per-request state on a
pooled engine; the CLR exception behind an interop error is reachable
through `JintException.TryGetClrException` with opt-in
`ChainClrExceptions()`, and a host method's own `TargetException` is no
longer mistaken for a receiver mismatch; and a recursion-limit failure
propagates out of a module load instead of becoming a catchable
rejection.

**Performance, gated.** Against v4.15.3 on idle hardware, medians of
three paired runs: `controlflow-recursive` **−15.6% time and −40.4%
allocation** (proper tail calls), `bitops-3bit-bits-in-byte` −8.9%,
`math-spectral-norm` −7.3%, `crypto-sha1` −6.9%, `3d-raytrace` −5.9%,
`math-cordic` −5.8%, with a broad −1–4% tail across the call- and
string-heavy rows; no row moved outside its own measured cross-run
envelope in the other direction, and allocation is flat within ±0.2%
suite-wide. Warmed `parseInt` call sites take the frameless fast-call
lane (−13% on the parse loop), joined by the `Number` predicates,
`String.prototype.indexOf`/`startsWith`/`endsWith`/`includes`/`at`/`substr`,
global `isNaN`/`isFinite` and `Array.isArray` (−3% to −19%) and the
`Map`/`Set` method family (`map.get` hit loop −13%); existence questions
on a wrapped dictionary answer from `ContainsKey`, taking `in` −33% with
−98% allocation and `Object.keys` −37%; resolving an inherited global no
longer allocates per miss (−99.99% on the read loop) and a global
created through an inherited write keeps the in-place store; JSON
replacer/reviver eligibility is decided once per document, built-in
callback dispatch once per loop, a call site's arguments reach an
interpreted callee in registers, and function-local `let`/`const` live
in fixed slots.

**Breaking changes.**
`Int32Extensions`/`Int64Extensions`/`DoubleExtensions` — polyfill hosts
that leaked into the public API — are now internal; on
net462/netstandard2.0, code with `using Jint;` may have bound span
`Parse`/`TryParse` members through them. `JsonParser` rejects trailing
commas. `Number.parseInt.length`/`Number.parseFloat.length` report their
spec values. Post-construction mutation of an `Options` instance no
longer reaches an already-built engine, and `Options.Configure`
callbacks work again. `UnwrapIfPromise` reports a cancelled engine as
`ExecutionCanceledException` instead of a timeout. Time-zone matching is
ASCII-case-insensitive per ECMA-402.

On the [engine comparison
benchmarks](https://github.com/sebastienros/jint/blob/main/Jint.Benchmark/README.md),
Jint 4.16.0 is the fastest engine outright on 5 of 12 scripts — leading
`dromaeo-object-regexp-modern` over native V8 by 1.25× — in a
statistical tie for first on `interop-collection-traversal`, the fastest
managed engine on 10 of 12, the fastest interpreter on all 12, and
8.6×–11.2× ahead of ClearScript (native V8) on every interop row while
allocating 3.9×–12.4× less than the nearest managed competitor.

## What's Changed
* Run the repository's own host tests under Release-mode contract
verification by @​lahma in
sebastienros/jint#2866
* Update test262 suite and implement Iterator.prototype.join by @​lahma
in sebastienros/jint#2867
* Stop a closing iterator from swallowing the error that closed it by
@​lahma in sebastienros/jint#2868
* Give every benchmark row its own engine by @​lahma in
sebastienros/jint#2873
* Hand a call site's arguments to an interpreted callee in registers by
@​lahma in sebastienros/jint#2874
* Ask once per loop, not once per element, how to call a built-in's
callback by @​lahma in sebastienros/jint#2876
* Say which spec document to read for a feature by @​lahma in
sebastienros/jint#2882
* Run PR CI on every pull request, not only those targeting main by
@​lahma in sebastienros/jint#2883
* Update test262 suite and adopt the new take/drop RangeError by @​lahma
in sebastienros/jint#2877
* Implement Iterator Chunking by @​lahma in
sebastienros/jint#2878
* Say to write against the modern BCL and polyfill downwards by @​lahma
in sebastienros/jint#2884
* Implement Iterator Includes by @​lahma in
sebastienros/jint#2879
* Ask once per document, not once per key, how to call a JSON replacer
or reviver by @​lahma in sebastienros/jint#2885
* Use double.IsFinite instead of hand-rolled NaN and infinity pairs by
@​lahma in sebastienros/jint#2880
* Bump the testing group with 1 update by @​dependabot[bot] in
sebastienros/jint#2889
* Stop charging closures for per-engine lazy globals by @​lahma in
sebastienros/jint#2890
* Bump the analyzers group with 1 update by @​dependabot[bot] in
sebastienros/jint#2888
* Make the lazy-global re-arm on restore a contract instead of an
accident by @​lahma in sebastienros/jint#2892
* Let the amortized constraint cadence span top-level entries into the
engine by @​lahma in sebastienros/jint#2886
* Let a function's own let/const live in its fixed slots by @​lahma in
sebastienros/jint#2887
* Keep a host function in the engine's own realm after a second realm
exists by @​lahma in sebastienros/jint#2893
* Make an engine's principal realm [[HostDefined]] reachable by @​lahma
in sebastienros/jint#2891
* Carry a labelled break/continue target on the completion record by
@​lahma in sebastienros/jint#2894
* Make Array.prototype.sort stable on every target framework by @​lahma
in sebastienros/jint#2898
* Stop toSorted and %TypedArray%.sort hanging on an inconsistent
comparator by @​lahma in sebastienros/jint#2899
* Delete the evaluation context's dead completion channel by @​lahma in
sebastienros/jint#2895
* Stop shipping the numeric polyfill hosts as public API by @​lahma in
sebastienros/jint#2901
 ... (truncated)

Commits viewable in [compare
view](sebastienros/jint@v4.15.3...v4.16.0).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=Jint&package-manager=nuget&previous-version=4.15.3&new-version=4.16.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>
This was referenced Aug 19, 2026
Closed
Closed
Merged
Closed
Merged
Closed
Closed
Closed
Closed
Merged
Merged
Closed
Closed
Closed
This was referenced Aug 27, 2026
Closed
Closed
Closed
This was referenced Aug 30, 2026
Closed
Closed
Merged
Closed
Open
This was referenced Sep 6, 2026
Closed
Closed
Open
Closed
Closed
Closed
This was referenced Sep 13, 2026
Closed
Closed
Closed
Closed
Open
Open
Open
This was referenced Sep 20, 2026
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lahma