Skip to content

shell: don't consume the next word as a file target for 2>&1 / 1>&2 - #34904

Closed
robobun wants to merge 2 commits into
mainfrom
farm/d3e87e80/shell-dup-redirect-no-file-operand
Closed

robobun wants to merge 2 commits into
mainfrom
farm/d3e87e80/shell-dup-redirect-no-file-operand

Conversation

@robobun

@robobun robobun commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator

Repro

import { $ } from "bun";
await $`${{ raw: "echo a 2>&1 b" }}`.quiet();
// bash : prints "a b" to stdout
// bun  : writes "a\n" into a file named `b`

1>&2 had the mirrored problem: echo a 1>&2 b created an empty file b and wrote a to stdout instead of writing a b to stderr.

Cause

parse_redirect in src/shell_parser/parse.rs unconditionally read a file operand after consuming a Redirect token. The duplicate_out() check was only reached when parse_atom() returned None (i.e. the redirect was the last token of the command), so for echo a 2>&1 b it grabbed b and stored it as redirect_file.

Fix

Check redirect.duplicate_out() before attempting to parse an operand; fd-dup redirects carry their target fd in the token itself and never take a file.

parse_simple_cmd also now interleaves argument and redirect parsing, so the b that follows the dup stays in name_and_args instead of starting a new statement. That loop is the same shape as #34901 (which fixes the > file variant of the same interleaving problem), so whichever lands first the other is a trivial merge; the parse_redirect change here is the new piece.

A second redirect after the dup (e.g. echo a 2>&1 b > f) now reports the existing "Multiple redirects are not supported yet" parse error, which previously surfaced as the less helpful expected a command or assignment but got: "Redirect".

Verification

  • test/js/bun/shell/parse.test.ts: echo a 2>&1 b and echo a 1>&2 b parse as one command with name_and_args: [echo, a, b] and redirect_file: null; bare echo a 2>&1 is unchanged.
  • test/js/bun/shell/file-io.test.ts: both forms print a b to the expected stream and no file b is created.

Both fail on the released binary (parse sees redirect_file: b; behavioral sees empty stdout and a file on disk). Full parse.test.ts, lex.test.ts, file-io.test.ts, and bunshell.test.ts (414 tests) pass on this branch.


[review] gate passed · iteration 0 · 4 files touched

fails on main (without fix)
ASAN without fix: 4 failed, 4 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/shell/file-io.test.ts test/js/bun/shell/parse.test.ts
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (b6ee4df4d)

test/js/bun/shell/file-io.test.ts:
(pass) IOWriter file output redirection > basic file redirection > simple echo to file [83.91ms]
(pass) IOWriter file output redirection > basic file redirection > empty output to file [9.72ms]
(pass) IOWriter file output redirection > basic file redirection > zero-length write should trigger onIOWriterChunk callback [6.96ms]
207 | 
208 |     async doChecks(stdout: Buffer, stderr: Buffer, exitCode: number): Promise<void> {
209 |       const tempdir = this.tempdir || "NO_TEMP_DIR";
210 |       if (this.expected_stdout !== undefined) {
211 |         if (typeof this.expected_stdout === "string") {
212 |           expect(stdout.toString()).toEqual(this.expect
... (truncated)

release without fix: 5 failed, 4 skipped
bun test v1.4.0-canary.1 (1498d7b77)

test/js/bun/shell/file-io.test.ts:
(pass) IOWriter file output redirection > basic file redirection > simple echo to file [1.51ms]
(pass) IOWriter file output redirection > basic file redirection > empty output to file [0.22ms]
(pass) IOWriter file output redirection > basic file redirection > zero-length write should trigger onIOWriterChunk callback [0.20ms]
207 | 
208 |     async doChecks(stdout: Buffer, stderr: Buffer, exitCode: number): Promise<void> {
209 |       const tempdir = this.tempdir || "NO_TEMP_DIR";
210 |       if (this.expected_stdout !== undefined) {
211 |         if (typeof this.expected_stdout === "string") {
212 |           expect(stdout.toString()).toEqual(this.expected_stdout.replaceAll("$TEMP_DIR", tempdir));
                                          ^
error: expect(received).toEqual(expected)

- "a b
- "
+ ""

- Expected  - 2
+ Received  + 1

      at doChecks (/workspace/bun/test/js/bun/shell/test_builder.ts:212:37)
      at run (/workspace/bun/test/js/bun/shell/test_builder.ts:251:20)
      at async <anonymous> (/workspace/bun/test/js/bun/shell/test_builder.ts:297:24)
(fail) IOWriter file output redirec
... (truncated)
passes on PR (with fix)
ASAN with fix: 4 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/shell/file-io.test.ts test/js/bun/shell/parse.test.ts
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (b6ee4df4d)

test/js/bun/shell/file-io.test.ts:
(pass) IOWriter file output redirection > basic file redirection > simple echo to file [59.26ms]
(pass) IOWriter file output redirection > basic file redirection > empty output to file [8.00ms]
(pass) IOWriter file output redirection > basic file redirection > zero-length write should trigger onIOWriterChunk callback [6.39ms]
a b
(pass) IOWriter file output redirection > fd-dup redirect followed by a word > 2>&1 does not consume the next word as a file [9.82ms]
a b
(pass) IOWriter file output redirection > fd-dup redirect followed by a word > 1>&2 does not consume the next word as a file [14.79ms]
(pass) IOWriter file output redirection > drainBufferedDat
... (truncated)

release with fix: 4 skipped
$ bun scripts/build.ts --profile=release
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     b6ee4df4d4
  features     (none)

22 deps, 106 codegen, 1169 objects in 939ms

ninja: Entering directory `/workspace/bun/build/release'
[1/1232] install /workspace/bun
bun install v1.4.0-canary.1 (1498d7b77)

Checked 124 installs across 170 packages (no changes) [34.00ms]
[2/1232] gen ErrorCode+*.h
[3/1232] install /workspace/bun/packages/bun-error
bun install v1.4.0-canary.1 (1498d7b77)

Checked 1 install across 2 packages (no changes) [2.00ms]
[4/1232] install /workspace/bun/src/node-fallbacks
bun install v1.4.0-canary.1 (1498d7b77)

Checked 129 installs across 147 packages (no changes) [7.00ms]
[5/1232] gen .bind.ts → GeneratedBindings.cpp
[6/1232] fetch zlib
[zlib] up to date
[7/1232] fetch libjpeg-turbo
[libjpeg-turbo] up to d
... (truncated)
diff hotspot
src/runtime/shell/Builtin.rs      |  4 ++--
 src/shell_parser/parse.rs         | 34 ++++++++++++++++++++++++++--------
 test/js/bun/shell/file-io.test.ts | 19 +++++++++++++++++++
 test/js/bun/shell/parse.test.ts   | 29 +++++++++++++++++++++++++++++
 4 files changed, 76 insertions(+), 10 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                               reads  edits  tests
src/runtime/shell/Builtin.rs           1      1      0
src/shell_parser/parse.rs              8      3      0
test/js/bun/shell/file-io.test.ts      1      1      0
test/js/bun/shell/parse.test.ts        1      2      0

root cause · written by the author bot

The shell parser's parse_redirect unconditionally called parse_atom() to read a file operand after any redirect token, so for fd-duplication forms like 2>&1 that take no operand, the following word was wrongly consumed as the redirect target instead of remaining a command argument. The fix checks redirect.duplicate_out() immediately after extracting the redirect flags and returns early with no file operand, leaving subsequent words to be parsed as arguments. It also reports the existing "multiple redirects are not supported" error when a dup redirect appears alongside another redire…

`2>&1` / `1>&2` are complete on their own; the following word is an
argument of the command. `parse_redirect` was only checking
`duplicate_out` after `parse_atom()` had already consumed the next
word, so `echo a 2>&1 b` opened a file named `b` and wrote "a\n"
into it instead of printing "a b".

Check `duplicate_out` before reading a file operand, and interleave
argument and redirect parsing in `parse_simple_cmd` so the word that
follows the dup stays in `name_and_args`.
@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9a62de6b-911a-47c4-9545-b1f3a624de3a

📥 Commits

Reviewing files that changed from the base of the PR and between 5b98630 and b6ee4df.

📒 Files selected for processing (4)
  • src/runtime/shell/Builtin.rs
  • src/shell_parser/parse.rs
  • test/js/bun/shell/file-io.test.ts
  • test/js/bun/shell/parse.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator Author

Reproduced with:

bun -e 'await Bun.$`${{raw:"echo a 2>&1 b"}}`.quiet().then(r => console.log(JSON.stringify(r.stdout.toString())))'

On 1.4.0-canary this prints "" and creates a file b containing a\n; on this branch it prints "a b\n" and no file is created.

@robobun

robobun commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:48 AM PT - Jul 21st, 2026

✅ @robobun, your commit b6ee4df4d4c22dfe6cbabaabf39dd48a184a6070 passed in Build #76773! 🎉


🧪   To try this PR locally:

bunx bun-pr 34904

That installs a local version of the PR into your bun-34904 executable, so you can run:

bun-34904 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — targeted parser fix with parse-level and behavioral coverage.

What was reviewed:

  • parse_redirect: moving the duplicate_out() early-exit before the operand read is correct; the lexer encodes the target fd in the flags so no file operand exists.
  • parse_simple_cmd loop: verified check() is a peek and ParsedRedirect::default() matches the old no-redirect path, so commands without redirects and echo a 2>&1 (bare) parse identically; the second-redirect guard turns a confusing leftover-token error into the existing "Multiple redirects" message.
  • Confirmed the parse test's {stdout: true, duplicate_out: true} for 2>&1 matches the lexer (which flips STDERR→STDOUT), and the subshell parse_redirect caller is unaffected since subshell redirects already error as unsupported.
Extended reasoning...

Overview

Two changes to src/shell_parser/parse.rs:

  1. parse_redirect now checks redirect.duplicate_out() before trying to consume a file operand, so 2>&1 / 1>&2 never eat the following word.
  2. parse_simple_cmd restructures its arg-collection into a loop that interleaves parse_atom and (at most one) parse_redirect, so a word after the redirect returns to name_and_args instead of being left as a stray token for the caller.

Tests added in parse.test.ts (AST shape for echo a 2>&1 b, echo a 1>&2 b, and unchanged bare echo a 2>&1) and file-io.test.ts (end-to-end: correct stream, correct output, .doesNotExist("b") negative contract).

Security risks

None. Pure parser control flow over already-tokenized input; no allocation-size arithmetic, no FFI, no filesystem writes in the parser path. The fix removes an unintended file write.

Level of scrutiny

Medium. The parse_simple_cmd loop is a real control-flow change affecting all commands with redirects, but it is ~15 lines, the no-redirect path is provably identical (ParsedRedirect::default() = empty flags + None, same as the old parse_redirect() fallthrough when has_redirect was false), and the single-redirect path only differs by continuing to collect args afterward — which is the bash-correct behavior. The other parse_redirect caller (subshell, line 1236) is guarded by an "unsupported" error on any non-empty redirect flags, so no behavior change there.

Other factors

  • I cross-checked the test's expected flags against the lexer at lines 3354-3372: 2>&1 produces STDOUT|DUPLICATE_OUT (STDERR is removed), matching redirect({ stdout: true, duplicate_out: true }). The doc comment on DUPLICATE_OUT (lines 589-590) is inverted, but that's pre-existing and not touched here.
  • .stderr("") in the new TestBuilder cases follows the established in-process shell-buffer convention used ~95× across test/js/bun/shell/.
  • The overlap with #34901 is called out in the description; the loop shape is compatible.
  • PR reports full parse.test.ts, lex.test.ts, file-io.test.ts, and bunshell.test.ts passing.

…T comments

The stored bit is the *target* fd: `2>&1` emits stdout=true and
`1>&2` emits stderr=true. The doc comment on the flag and a matching
annotation in Builtin.rs had them swapped.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — focused parser fix with matching parse-level and behavioral tests.

What was reviewed:

  • Verified against the lexer (parse.rs:3354-3369) that 2>&1 → stdout+duplicate_out and 1>&2 → stderr+duplicate_out, so the corrected comments in Builtin.rs / parse.rs and the init_redirections branch semantics all line up.
  • Traced the new interleaved loop in parse_simple_cmd: check() peeks, parse_redirect() consumes; a trailing word after a dup redirect lands back in name_and_args, and a second Redirect token hits the new multi-redirect error.
  • Checked the other parse_redirect caller (subshell, parse.rs:1236) — the early duplicate_out() return is correct there too. ParsedRedirect derives Default, so the no-redirect path is unchanged.
  • Tests cover both directions, the bare 2>&1 case, .doesNotExist("b"), and the multi-redirect error message.
Extended reasoning...

Overview

Fixes a shell-parser bug where 2>&1 / 1>&2 incorrectly consumed the following word as a redirect file target. Two logic changes in src/shell_parser/parse.rs: (1) parse_redirect now checks redirect.duplicate_out() before attempting to parse a file operand instead of only in the parse_atom() == None fallback, and (2) parse_simple_cmd interleaves argument and redirect parsing so words after a redirect stay in name_and_args. Two comment-only hunks in Builtin.rs and the RedirectFlags doc fix inverted flag descriptions. Tests added in parse.test.ts (AST shape) and file-io.test.ts (end-to-end via TestBuilder).

Security risks

None. This is grammar handling in the shell parser; no auth, crypto, path traversal, or untrusted-length arithmetic is touched. The change removes an unintended file-creation side effect.

Level of scrutiny

Moderate — the shell parser is user-facing runtime code, but the diff is ~25 lines of localized control-flow change with a clearly identified root cause and a mechanical fix (move an existing guard earlier; loop instead of sequence). I cross-checked the comment corrections and test expectations against the lexer's actual flag assignment at parse.rs:3354-3369, which confirms 2>&1 produces STDOUT|DUPLICATE_OUT (STDERR removed) — the old comments were indeed inverted and the init_redirections branch in Builtin.rs (if redirect.stdout() { stderr = stdout.dup_ref() }) does the right thing.

Other factors

  • The interleaved-loop shape is acknowledged in the PR as overlapping #34901 for the > file word case; I grepped test/js/bun/shell for existing tests exercising a word after a file redirect and found none, so no silent behavior change to existing coverage.
  • ParsedRedirect has #[derive(Default)] (parse.rs:2152), so the no-redirect path yields the same empty flags/None as before.
  • The subshell caller of parse_redirect at parse.rs:1236 also benefits from the early duplicate_out() return — (cmd) 2>&1 word won't grab word either (though the subshell path doesn't loop back for more args, that's pre-existing and out of scope here).
  • The new "Multiple redirects" error is a strict UX improvement over the previous expected a command or assignment but got: "Redirect" and is covered by a test.
  • PR description states full parse.test.ts, lex.test.ts, file-io.test.ts, and bunshell.test.ts pass; robobun reproduced the fix on canary vs. this branch.

@robobun

robobun commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: this is covered by #34901. That PR has the same parser change, and it now carries the tests and the corrected DUPLICATE_OUT comments from this one.

@robobun robobun closed this Oct 1, 2026
robobun added a commit that referenced this pull request Oct 1, 2026
…OUT comments

`2>&1` sets STDOUT with DUPLICATE_OUT and `1>&2` sets STDERR with
DUPLICATE_OUT. The comments on the flag and in the builtin redirect
setup said the reverse.

parse_simple_cmd now reports a second redirection, so the note in
parse_redirect that asked for that check is removed.

Tests: the fd-dup forms create no file named by the next word and do
not overwrite an existing one, `1>&2` and a trailing `2>&1` keep
their AST shape, `echo a 2>&1 b > f` reports the multiple-redirect
error, and the tail of an interpolated array after `>` stays in argv.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant