Skip to content

parser: carry the a ? b : c mark into arrow function bodies and assignments - #47133

Draft
robobun wants to merge 4 commits into
robobun/d526420d/stack-base-4from
robobun/d526420d/conditional-arrow-bodies-2
Draft

robobun wants to merge 4 commits into
robobun/d526420d/stack-base-4from
robobun/d526420d/conditional-arrow-bodies-2

Conversation

@robobun

@robobun robobun commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Depends on #45743, #45939 and #46209 (the base branch holds them). Draft until they have a decision.

Problem

  • bun -e 'const f = 1 ? v => ({ v }) : v => 0' fails with Expected ":" but found ";". It is legal JavaScript (bun -e and stdin use the TypeScript parser). In a .ts file, byId ? (n): number => (n) : n => n + 1 prints byId ? n : (number) => (n) => n + 1 with no error. tsc and Node 26 take both.
  • Between ? and :, (c) : d => e is an arrow function only if a second : follows (parser: accept TypeScript arrow return types inside conditionals, export type before a newline, and import type from #40818). The mark for that, EFlags::AfterQuestionAndBeforeColon, stops at the first token of the branch: the body of v => and the right side of q = do not get it.

Fix

  • parse_arrow_body and the 16 assignment operators pass the mark on, as tsc passes allowReturnTypeInArrowFunction.
  • As in tsc, (), (...a, (a: and (a? are parameters at first sight, and their body is a new expression.
  • Verified: test/bundler/transpiler/transpiler.test.js: 12 tests fail on the base. Two compare 4,000 generated conditionals with typescript.
  • Self-reviewed: 36 concerns raised, 31 addressed. Notes has the other five.

Background

  • a ? (b) : c => d is ambiguous: (b): c => d can be an arrow function with the return type c. The parser tries that on a snapshot (the attempt) and keeps it if a : follows.
  • tsc's flag starts again inside brackets, in a block body and in a first sight arrow function.
  • Weighed: the mark only behind name => (one shape of four). No first sight rule: accepted inputs print another program.

Downsides

  • The reading of tsc replaces the reading of main: a ? (b) : c => (d): T => e is now a ? (b) => d : (T) => e, not a ? b : (c) => (d) => e. bun fmt --arrow-parens=avoid writes such text.
  • Errors now, as in tsc: a ? (b) : c => async (d): Promise<T> => e.
  • Cost: an attempt parses its body twice, now also in arrow bodies and assignments. Nested 640 deep, a ? q => (p): T => … : e takes 330 ms and 405 MB (base: 12 ms, 33 MB). Other .ts statements: up to 109 more instructions. .ts nests 2.7% less deep.
Notes

Where this comes from

A fuzz-ledger note, relayed through a maintainer's account: two programs that tsc accepts. This PR is the first. The second is #45743. There is no GitHub issue and no user report for Bun. biomejs/biome#9548 reports the first line of the Problem for another tool, with code from a project: isFirstMount ? i => ({ [CONTENT_SLOT]: i }) : i => wrapSlotExpr(newExprs[i]). Main refuses that code in a .ts file too.

The trigger is narrow. A text that can be parameters ((c), ({ c }), (c, d), (c = d), ([c])) stands directly before the : as an arrow function body, as the right side of an assignment or as a nested false branch. And the false branch starts with a parameter without parentheses: name =>. With (name) =>, main already reads both lines of the Problem as tsc does. The transpiled output of 3,904 TypeScript and 7,999 JavaScript files of this repo is the same before and after.

The assignments, async, type parameters and the first sight rule are not in the note. They are the rest of the same tsc rule, found with the generated inputs below.

Inputs

input main (7942008) and the base this PR tsc 6.0.2
x = byId ? id => ({ id }) : name => ({ name }); Expected ":" but found ";" x = byId ? (id) => ({ id }) : (name) => ({ name }); the same tree
x = byId ? (n): number => (n) : n => n + 1; x = byId ? n : (number) => (n) => n + 1;, no error x = byId ? (n) => n : (n) => n + 1; the same tree
x = a ? q = (c) : d => e; Expected ":" but found ";" x = a ? q = c : (d) => e; the same tree
x = a ? (b): T => q = (c) : d => e; x = a ? b : (T) => q = (c) => e;, no error x = a ? (b) => q = c : (d) => e; the same tree
x = a ? <T>(b) : c => d; Expected ":" but found ";" x = a ? b : (c) => d; the same tree
x = a ? b => async() : c => d; Expected ":" but found ";" x = a ? (b) => async() : (c) => d; ':' expected.
x = a ? (b) : c => () => (d) : e => f; x = a ? b : (c) => () => (d) => f; the same the same tree
x = a ? (b) : c => (d): T => e; x = a ? b : (c) => (d) => e; x = a ? (b) => d : (T) => e; the same tree as this PR
x = a ? (b) : c => async (d): Promise<T> => e; x = a ? b : (c) => async (d) => e; Expected ";" but found "=>" ';' expected.
x = a ? () => (b) : c => d; Expected ":" but found ";" the same ':' expected.

TypeScript has ten tests for this rule, parserArrowFunctionExpression8 to 17. Main prints tsc's tree for seven of them and this PR for ten. A test holds the ten lines.

The rule

tsc hands allowReturnTypeInArrowFunction down where an expression continues the branch: into an arrow function body, into the right side of an assignment and into the false branch of a nested conditional. It starts again with true inside parentheses, brackets, call arguments and a block body.

EFlags is that flag here. It has three marks.

  • AfterQuestionAndBeforeColon: directly between ? and :, as on main. (c) : d => e is an arrow function only if the attempt finds a : behind its body. A call async(...) stays a call if the attempt fails.
  • AfterQuestionAndBeforeColonCarried: the same in an arrow function body or on the right side of an assignment. Here async() before a : is a call only if no reading leaves a : for the conditional. a ? q => async() : d => e is a call. a ? q => async (): T => (d) : e => f has such a reading, so it stays the error that it is on main and in tsc 6.0.
  • AfterQuestionAndBeforeColonInTypedArrow: the same, in the body of an arrow function whose own return type is being tried. An attempt around it waits for a :, so this state follows tsc exactly: (), (...a, (a: and (a? start an arrow function at once, also behind async.

A head that tsc decides at first sight (isParenthesizedArrowFunctionExpression returns Tristate.True) has a new expression as its body: the mark stops there. a ? (b) : c => () => (d) : e => f needs that. tsc reads () => (d): e => f as the body of c =>, so no : is left for (b). With the mark in the body of () =>, (d) ends the body and (b): c => () => (d) becomes an arrow function.

<T>( and async <T>( get the mark too. tsc never decides them at first sight. <T>(b) can be a cast, so it needs the attempt. <T>(b: B), <T>() and <T>(...b) are no cast: they take no attempt, as on main, so an error in their body is reported where it is. In .tsx, <T,>( is decided at first sight, so it gets no mark.

The memo of attempts is keyed by the position of the :, by allow_await and now by whether the body is read under a mark. The same : can be reached with a first sight head and with another head.

What a build without the first sight rule does

On an earlier build of this change, with the rule switched off: of 150,000 generated .ts conditionals, 418 that tsc accepts printed another program and 67 failed. With the rule: none. The tests under "Such a body takes ..." fail without it.

Observable changes

Counted on 360,000 generated conditionals (300,000 .ts, 60,000 .tsx) with release builds of the base and this PR. The false branch starts with a bare name => in more than half of them. tsc 6.0.2 accepts 206,076.

  • The same tree as tsc: 176,052 on main, 184,004 on the base, 206,069 with this PR.
  • tsc's tree now: 19,881 inputs that were errors, and 2,184 that printed another program with no error.
  • The 7 that are left have a default value inside a function type: a ? async ({ b }) : (b: B = c) => d => c : e. tsc's parser takes that and its checker rejects it. The type skipper here does not take it. That is not part of this change.
  • Accepted now, tsc rejects: 6,133. 5,913 of them have a call of async, which stays a call in an arrow function body as main keeps a ? async() : 0. tsc 6.0.2 rejects each async() before a :.
  • An error now, tsc rejects too: 177. tsc gives the first : to the parentheses, and what is left is not a false branch: a ? (b) : c => (d): T[] => e, a ? (b) : c => async (d): Promise<T> => e, a ? (b) : c => (d, e?): T => f. Main reads c => ... as the false branch.
  • Another program than on the base, tsc rejects: 119.
  • Another error message: 805.
  • An error now, tsc accepts: 0.

With the js loader, the 300,000 .ts inputs give the same output or error message as before.

Two more changes on inputs that tsc rejects:

  • A return type that is not a type. Main reports Unexpected => for a ? (b): => c : d, because the attempt reads the type again with errors on. The body of q => had no attempt, so main prints a ? q => (b): => c : d as an arrow function. Now it is an error there too.
  • bun fmt --arrow-parens=avoid, the setting of this repo, writes a ? (b = 1) : c => (d): number => e from a ? b = 1 : (c) => (d): number => e. Prettier 3.9.9 writes the same. tsc and Node 26 read the written text as this PR does: a ? ((b = 1): c => (d)) : (number => e). Main reads it as the text before the formatter. The formatter is the defect there.

Known gaps

  • Three test lines pin inputs that tsc and main accept and this PR does not read as they do: two are errors, and one prints another program. A source text uses await as a name before /, inside parentheses in the body of a ? async (h, k) : R =>. tsc reads such parentheses as parameters first, where await is a name and / divides. The attempt reads an expression, where await /x.../ is a regular expression, so the parentheses end at another place.
  • a ? () => (b) : c => d stays an error, as in tsc 6.0.2 and on main: () starts an arrow function at first sight, and its body is a new expression. It is legal JavaScript.
  • fix(64231): avoid parsing async() calls as typed arrows in conditionals microsoft/TypeScript#64233 (open) makes tsc accept a ? async() : 0. Its test has 85 lines. This PR reads 54 as that fix does (main: 44). A test holds 38 of them. 16 are errors here, as in tsc 6.0.2. 11 print another tree, 10 of them as on main: a ? async() : T => async (): T => 0 keeps the call. 4 are error recovery lines.

Cost of the attempt

An attempt parses the return type and the body on a snapshot, restores, and the caller parses them again. What a discarded attempt allocated in the arena stays until the file is done. Main has this cost at the first token of a branch. This PR has it in arrow function bodies, on the right side of assignments and behind <T> too.

One transformSync per process, release builds, peak RSS. An idle process is about 30 MB.

nested conditionals levels main base this PR
a ? q => (p): T => … : e 320 / 640 5 ms, 29 MB / 14 ms, 34 MB 3 ms, 30 MB / 12 ms, 33 MB 56 ms, 123 MB / 327 ms, 405 MB
a ? q => (p) : c => … 320 / 640 error error 52 ms, 116 MB / 231 ms, 375 MB
a ? q = (p): T => … : e 320 / 640 5 ms, 29 MB / 13 ms, 32 MB 3 ms, 30 MB / 7 ms, 32 MB 44 ms, 89 MB / 161 ms, 267 MB
a ? q = (p) : c => … 320 / 640 error error 38 ms, 81 MB / 255 ms, 237 MB
a ? <T>(p): T => … : e 320 / 640 2 ms, 29 MB / 10 ms, 32 MB 2 ms, 30 MB / 6 ms, 31 MB 60 ms, 87 MB / 171 ms, 258 MB
a ? (p = …): T => p : e 320 / 640 9 ms, 29 MB / 7 ms, 32 MB 2 ms, 30 MB / 4 ms, 32 MB 2 ms, 30 MB / 5 ms, 33 MB
a ? (b) : c => … : e 320 / 640 78 ms, 85 MB / 263 ms, 256 MB 39 ms, 86 MB / 151 ms, 257 MB 41 ms, 86 MB / 151 ms, 258 MB
a ? (b) : c => … 320 / 640 35 ms, 78 MB / 132 ms, 227 MB 34 ms, 79 MB / 142 ms, 227 MB 35 ms, 79 MB / 131 ms, 228 MB

640 levels of the first row are 14.7 KB. The times are single runs and move by a factor of two. The memory is stable and grows four times per doubling. Each attempt runs once: the tests have chains of 40, which 2^40 parses would not finish. There is no cap on depth or on attempts.

#46311 (draft) keeps the arrow function that an attempt finds, so a kept attempt is not parsed again. It does not change the rows with a discarded attempt.

Measurements

Release builds from clean trees, linux x64: the base 508fab2 (main 7942008 with #45743, #45939 and #46209) and 86460ad. The head 6f381e3 moves two helpers out of the generic impl. A release build of it has the same .text size, the same frames and the same result on the 360,000 generated inputs.

  • Instructions inside one transformSync call, per statement of a file with 20 statements (single-stepped, JIT off, median of 3 runs).
    • JavaScript: fewer in each of the 15 files measured. x = a; -12 of 6,016. x = (a); -20. x = a => a; -38. x = a ? b => c : d => e; -74.
    • TypeScript outside a conditional: x = a; +7 of 6,451. x = (a); +7. x = a => a; -5. x = (a: T): U => a; -1. x = <T[]>(a); +7. x = f<T>(a); +9. x = { a: (b), c: [d, (e)] }; +13.
    • TypeScript between ? and :: x = a ? b : c; +11 of 9,723. x = a ? b = c : d; +25. x = a ? (b) => c : d; +94 of 16,970. x = a ? (b) : c => d : e; +100. x = a ? (b: B): T => c : d; +84. x = a ? (b) : (c); +109 of 12,849. x = a ? b => c : d => e; -4.
    • x = a ? b => (c): T => d : e; +1,895 of 21,390. The body of b => has an attempt now.
  • Stack frames. parse_paren_expr in TypeScript: 1,088 B on main, 496 B on the base, 528 B with this PR (the scan-only copy: 528 to 544 B). JavaScript: 448 B on all three. parse_arrow_body: 192 to 176 B in JavaScript, 192 B in TypeScript. parse_suffix: 240 to 224 B in one copy and 224 to 240 B in another.
  • try_skip_type_script_arrow_return_type_with_backtracking is #[inline(never)] now. Without that, LLVM inlined it into parse_paren_expr for this change, with its lexer snapshot: 752 B per frame and 18% less nesting.
  • The deepest ((((1)))) that parses: 7,031 to 6,844 in .ts (2.7% less), 7,333 to 7,338 in .js. () => () => ... 1: 6,039 to 5,900 in .ts, 6,260 to 6,341 in .js. At a 1 MB stack in .ts: 750 to 732.
  • .text 83,513,897 to 83,515,177 B (+1,280). Parser symbols (nm -S): +1,315 B.

Tests

  • The pins: each sub-rule, the 16 assignment operators, .tsx, every Bun.Transpiler API, a spawned .ts file and bun -e, emitDecoratorMetadata behind a kept and a discarded attempt, and what a discarded attempt must leave behind.
  • TypeScript's parserArrowFunctionExpression8 to 17.
  • Two tests with typescript as oracle, for .ts and .tsx: 2,000 statements each from a generator with the seed 11. On main 247 and 150 of them differ from tsc. They are skipped on debug and ASAN builds, where loading TypeScript takes about a minute. The oracle test of parser: a : after (b as B) or <B[]>(b) is not an arrow function return type (TypeScript) #46209 gets a plain b item: 34 of its inputs differ on the base.
  • With the debug build: transpiler.test.js (279 pass), test/bundler/esbuild/ts.test.ts (59), test/cli/run/transpiler-cache.test.ts (24), runtime-transpiler.test.ts (15), decorators.test.ts (25), decorator-metadata.test.ts (5), es-decorators.test.ts (427), es-decorators-esbuild.test.ts (146), scope-mismatch-panic.test.ts (21), transpiler-stack-overflow.test.ts (1), test/bundler/bundler_regressions.test.ts (9). With release builds: transpiler.test.js has 12 failing tests on the base and 282 pass with this PR.

History and reference

Not in this PR

Self-review

A review of this change raised 33 concerns, and a second branch for the same two lines (see the comment below) gave three more. 31 are addressed:

Five are not:

Related

…ignments

In a TypeScript file, `a ? b => (c) : d => e` failed with `Expected ":" but
found ";"`, and `a ? (b): T => (c) : d => e` printed
`a ? b : (T) => (c) => e`. tsc accepts both.

The mark that `parse_paren_expr` reads between "?" and ":" now reaches the
body of an arrow function and the right side of an assignment, as
`allowReturnTypeInArrowFunction` does in tsc. Parentheses that tsc takes as
parameters at first sight ("()", "(...a", "(a:", "(a?") start a new
expression in their body.
@robobun

robobun commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 9:35 AM PT - Oct 28th, 2026

❌ @robobun, your commit 6f381e3 has 1 failures in Build #134413 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47133

That installs a local version of the PR into your bun-47133 executable, so you can run:

bun-47133 --bun

@robobun

robobun commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

How to reproduce on main (7942008):

  1. Run bun -e 'const f = 1 ? v => ({ v }) : v => 0; console.log(JSON.stringify(f(2)))'. Main fails with error: Expected ":" but found ";". This branch prints {"v":2}, as Node does.

  2. Write a.ts:

const byId = false;
const pick = byId ? id => ({ id }) : name => ({ name });
console.log(JSON.stringify(pick("a")));
const next = byId ? (n): number => (n) : n => n + 1;
console.log(typeof next, next(1));

Run bun a.ts. Main fails at line 2 with the same error. Without line 2 and 3, main prints function [Function]: it reads line 4 as byId ? n : (number) => (n) => n + 1. This branch prints {"name":"a"} and function 2.

The test: bun bd test test/bundler/transpiler/transpiler.test.js. 12 tests fail on the base branch and pass on this branch (4 of them are skipped on debug builds).

The branch is on robobun/d526420d/stack-base-4, which is main with #45743, #45939 and #46209.

CI on the head 6f381e3 (build 134413): 177 jobs passed. The four darwin test-bun jobs got no agent and expired in the queue. On darwin 27 aarch64, the download of the build timed out before a test ran. Each test that failed on another lane passed on its retry, and none of them is a parser test.

PR: #47133

@robobun

robobun commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

A second branch for the same two lines exists: robobun/6e7e8a46/conditional-arrow-bodies. It started from the first form of this carry (39286f4). This PR owns the change, so that branch gets no PR.

The two parsers print the same for 405,464 generated conditionals (tsc 6.0.2 as the reference). The branch differs in three places. Each was checked on a release build with the parser of 5e78a89.

  1. A legacy decorator behind a cast (the cast operand rule, now in parser: a : after (b as B) or <B[]>(b) is not an arrow function return type (TypeScript) #46209). With experimentalDecorators, class C { @<any>new Foo ['k']() {} } fails with Expected identifier but found "{". @<any>new Foo ['k'] with m() {} on the next line decorates m with new Foo["k"], with no error. Main decorates k with new Foo in both cases. The cause: the cast passes EFlags::TsCastOperand in place of TsDecorator to its operand. The change and a test are in 89fa9ad (parse_prefix.rs, test a legacy decorator behind a <T> cast ends before the name of its member).
  2. async (): T => (d) in an arrow function body or on the right side of an assignment. a ? q => async (): T => (d) : e => f is an error on main and in tsc 6.0.2. With this PR it prints a ? (q) => async() : (T) => (d) => f with no error. The same applies to a ? q = async (): T => (d) : e => f, a ? q += async (...b): T => (d) : e => f and a ? q => async() : T => (b): number => b. Node reads the first line as an async arrow function. The branch reads a call in these positions only when no reading leaves a : for the conditional, so a ? q => async() : d => e stays a call and the four lines stay errors.
  3. The position of an error. a ? <T>(b: B): U => { return (c; } : d reports Unexpected ":" at the parameter. Main reports Expected ")" but found ";" at the error. a ? <T>(b: B): U => c reports Unexpected ":", and main reports Expected ":" but found ";". In a real parse the branch skips the attempt for parentheses that can only be parameters.

The branch also has a test that compares 2,268 inputs (18 heads, 7 tails, 22 positions) with the typescript package, and rows for yield, case, the lines above and three kinds of text that tsc refuses.

Two facts for the description:

…(a: A)` as on main

- In an arrow function body or on the right side of an assignment between
  "?" and ":", `async()` before a ":" is a call only if no reading
  leaves a ":" for the conditional. `a ? q => async (): T => (d) : e => f`
  stays the error that it is on main.
- Behind type parameters, parentheses that can only be parameters take
  no attempt. An error in `a ? <T>(b: B): U => { ... } : c` is reported
  where it is.
…f the generic impl

Neither reads `TYPESCRIPT` or `SCAN_ONLY`. As methods of `P` they were compiled once per parser copy.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant