Skip to content

Temporal and Intl parsing cannot fail because the machine was busy - #3486

Merged
lahma merged 1 commit into
sebastienros:mainfrom
lahma:temporal-parse-regex-budget
Aug 27, 2026
Merged

lahma merged 1 commit into
sebastienros:mainfrom
lahma:temporal-parse-regex-budget

Conversation

@lahma

@lahma lahma commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Fixes #3485.

main is red because of this, but the red CI leg is the smaller half.

What an embedder sees today

Temporal.PlainDate.from('2024-01-15T10:30:00') — a correct call with a valid string — can throw
System.Text.RegularExpressions.RegexMatchTimeoutException. Built against a Jint whose only change is a
shortened budget, standing in for the starvation:

--- 1. can SCRIPT catch it? ---
   *** escaped Engine.Evaluate DESPITE the script's try/catch ***
   type    : System.Text.RegularExpressions.RegexMatchTimeoutException
   JintException?       False
   JavaScriptException? False

--- 2. what the HOST sees out of Engine.Evaluate ---
   message : The Regex engine has timed out while trying to match a pattern to an input string. This can
             occur for many reasons, including very large inputs or excessive backtracking caused by
             nested quantifiers, back-references and other factors.
   at System.Text.RegularExpressions.RegexRunner.<CheckTimeout>g__ThrowRegexTimeout|25_0()

It is not a JintException and not a JavaScriptException. Because RegexMatchTimeoutException is listed
in Throw.MustPropagateHostException, it goes straight through the script's own try/catch — a
script cannot defend itself — and reaches the host as a raw CLR exception whose message blames "very large
inputs or excessive backtracking" for a 19-character valid string.

Why it fires

Regex.MatchTimeout is a wall-clock deadline sampled while matching. It cannot tell "this pattern is
backtracking" apart from "this thread lost the CPU". Eight patterns in TemporalHelpers.cs and one in
IntlUtilities.cs carried 100 ms of it.

Driving TimeWithOffsetPattern with "10:30:00" — valid, 8 characters — 200,000 times on an ordinary
loaded machine, no induced load:

median 0.0101 ms
p99.9 0.061 ms
max 440.4 ms
max / median 43,607x

The maximum is 4.4x over the budget, on a valid input, from scheduling alone. The timeout is confirmed
sampled even on inputs this short: with a 1-tick budget, 10:30:00, 103000, 10:30:00+01:00 and
10:30:00.123456789+01:00:00 all throw.

It is not catastrophic backtracking

Best-of-200, scheduling noise stripped, adversarial near-miss inputs, infinite match timeout:

input length TimeWithOffset Instant Duration
1024 0.040 ms 0.037 ms 0.088 ms
4096 0.096 ms 0.098 ms 0.304 ms
16384 0.368 ms 0.380 ms 1.228 ms

16x the input costs 9–14x the time. Even 16 KB of adversarial input costs under 2 ms of CPU. There is
nothing here a 100 ms budget catches — raising it to 2 s would only make the false positive rarer.

One pattern was not linear, and this is the part that changed the fix. AnnotationPattern
(\[(!?)([^\]]+)\]) is quadratic on input holding no ]: each of the n start positions consumes the rest
of the string and backtracks over it. On net472 it exceeds 100 ms of genuine CPU at 16 KB — the new test
caught it against unfixed code with a real RegexMatchTimeoutException. It was unreachable with such input
only because its sole caller hands it InstantPattern's group 14, which that grammar already guarantees is
well-formed — an argument about the caller, not a property of the code.

The fix

Bounding the input length alone would not have fixed this: the failing input is short and valid, and the
exception fires because the thread was descheduled. And length cannot be capped uniformly — annotation
lists and duration digit runs are legitimately unbounded, so a small cap would reject valid input.

So the live budget goes, and the cost is bounded by construction instead:

  • Eight patterns in TemporalHelpers.cs and one in IntlUtilities.cs now take
    Regex.InfiniteMatchTimeout. The failure mode is removed rather than relabelled.
  • The fixed-width patterns reject anything longer than the longest string they could match, before
    matching. The bounds are exact — 13, 18, 16 and 37 — confirmed by reading the grammar and by searching
    each pattern's own language for its longest member, and pinned so a widened pattern cannot outgrow its
    cap silently.
  • AnnotationPattern is replaced by a linear walk, so the one superlinear pattern is gone rather than
    left unbounded. A test drives the walk and the retired pattern over ~5,000 inputs including the two cases
    the pattern's backtracking decided on its own: [], which has no content and is not an annotation, and
    [!], where the optional ! is given back to the content.

Options.Constraints.RegexTimeout never reached these patterns and still does not — it bounds
script-supplied regular expressions, unchanged (#3431 / §4.42).

Tests

The starvation cannot be reproduced without loading the machine, which would make the test the very flake
it is about — so TemporalParsePatternBudgetTests pins the property instead, with no timing assertion
anywhere
: no pattern carries a deadline, every cap is exactly the longest its pattern can match and
nothing longer matches (20,000 randomized probes per pattern), an over-long input is rejected as a
RangeError rather than a CLR exception, a valid string parses, and every pattern returns on 16 KB
adversarial input.

Verified failing against unfixed code first: NoInternalParsePatternCarriesAWallClockDeadline reports
Expected -1ms ... but found 100ms, and on net472
EveryPatternRejectsAdversarialInputWithoutBacktrackingAwayTheAfternoon dies with a real
RegexMatchTimeoutException on AnnotationPattern.

Verification

  • dotnet build -c Release — clean, TreatWarningsAsErrors on.
  • dotnet test -c Release — green: Jint.Tests 10,926 (net8.0/net10.0) and 7,550 (net472),
    Jint.Tests.PublicInterface 3,232 / 3,242 / 2,610, Jint.Tests.CommonScripts 28, Jint.Tests.SourceGenerators 71.
  • test262: 102,495 passed / 0 failed / 189 skipped — exactly the control.

One caveat worth recording: under full-solution parallel load this busy box also failed
intl402/supportedLocalesOf-unicode-extensions-ignored.js twice (35 s each). That is pre-existing — a
pristine main worktree at 5b2f8f1 fails the identical two cases under the same load, and both pass in
isolation. Not caused by this change, and arguably the same family of defect at a different site.

Migration guide

§4.52. Nothing an embedder configured changes; a host that caught RegexMatchTimeoutException around
Engine.Evaluate to absorb this can drop that handler.

@lahma
lahma force-pushed the temporal-parse-regex-budget branch from 5cf61a1 to 5818772 Compare August 27, 2026 18:48
The internal patterns that parse ISO 8601 strings for Temporal, and the one
that validates BCP 47 tags for Intl, each carried a 100 ms Regex.MatchTimeout.
That is a wall-clock deadline sampled while matching, so it cannot tell a
pattern that is backtracking apart from a thread that lost the CPU: a
descheduled thread trips it having burned almost no CPU at all.

Measured on an ordinary loaded machine, driving TimeWithOffsetPattern with the
valid 8-character string "10:30:00" 200,000 times: median 0.0101 ms, p99.9
0.061 ms, max 440 ms. The maximum is 4.4x over the budget, on a valid input,
from scheduling alone, and the timeout is confirmed to be sampled on inputs
that short. The same thing failed CI parsing a correct ISO string.

What that produced was not a JavaScript error. RegexMatchTimeoutException is
listed in Throw.MustPropagateHostException, so it escaped Engine.Evaluate as a
raw CLR exception, straight through the script's own try/catch -- a script
could not defend itself, and the message blamed "very large inputs or
excessive backtracking" for a 19-character valid string.

The patterns are linear, so the budget could only ever fire spuriously.
Best-of-200 over adversarial near-miss inputs with no timeout: 16 KB costs
0.37-1.23 ms of CPU, and 16x the input costs 9-14x the time. Raising the
budget would only make the false positive rarer, so the budget goes and the
cost is bounded by construction instead: the fixed-width patterns reject
anything longer than the longest string they could match without matching at
all, and those bounds are exact and pinned.

One pattern was not linear. AnnotationPattern -- \[(!?)([^\]]+)\] -- is
quadratic on input holding no ']', and exceeds 100 ms of genuine CPU at 16 KB
on net472. It was unreachable with such input only by an argument about its
caller, so it is replaced by a linear walk, pinned by a test that drives both
over a corpus including the two cases the pattern's backtracking decided on
its own ("[]" and "[!]").

The starvation cannot be reproduced without loading the machine, which would
make the test the very flake it is about, so the tests pin the property: no
pattern carries a deadline, every cap is the longest its pattern can match, an
over-long input is rejected as a JavaScript error, and a valid string parses.

Fixes sebastienros#3485

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S
@lahma
lahma force-pushed the temporal-parse-regex-budget branch from 5818772 to cfd5c3d Compare August 27, 2026 18:50
@lahma
lahma merged commit ba18893 into sebastienros:main Aug 27, 2026
7 checks passed
lahma added a commit that referenced this pull request Sep 1, 2026
…3486) (#3543)

The internal patterns that parse ISO 8601 strings for Temporal, and the one
that validates BCP 47 tags for Intl, each carried a 100 ms Regex.MatchTimeout.
That is a wall-clock deadline sampled while matching, so it cannot tell a
pattern that is backtracking apart from a thread that lost the CPU: a
descheduled thread trips it having burned almost no CPU at all.

What that produced was not a JavaScript error. RegexMatchTimeoutException is
listed in Throw.MustPropagateHostException, so it escaped Engine.Evaluate as a
raw CLR exception, straight through the script's own try/catch -- a script
could not defend itself, and the message blamed "very large inputs or
excessive backtracking" for a short valid string.

The patterns are linear, so the budget could only ever fire spuriously, and
the cost is bounded by construction instead: the fixed-width patterns reject
anything longer than the longest string they could match without matching at
all, and those bounds are exact and pinned.

One pattern was not linear. AnnotationPattern -- \[(!?)([^\]]+)\] -- is
quadratic on input holding no ']'. Measured on this branch against the net462
asset: 4 KB of '[' costs 305 ms of genuine CPU, 8 KB 951 ms, 16 KB 2.21 s and
32 KB 7.08 s, and with the shipped 100 ms budget every one of those throws
RegexMatchTimeoutException. It was unreachable with such input only by an
argument about its caller, so it is replaced by a linear walk -- 0.13 ms over
200,000 '[' -- pinned by a test that drives both over a corpus including the
two cases the pattern's backtracking decided on its own ("[]" and "[!]").

The starvation cannot be reproduced without loading the machine, which would
make the test the very flake it is about, so the tests pin the property: no
pattern carries a deadline, every cap is the longest its pattern can match, an
over-long input is rejected as a JavaScript error, and a valid string parses.

Backport of #3486 (main: ba18893). The engine change applies unmodified; the
test file is transcribed from NUnit to xUnit v3, which is what Jint.Tests uses
on this branch, and the v5 migration-guide entry is dropped because 4.x has no
such document.

Fixes #3485


Claude-Session: https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Temporal and Intl string parsing can fail with an uncatchable RegexMatchTimeoutException because the machine was busy

1 participant