Skip to content

fix(core): jsonl write([]) should leave an empty file, not a stray newline - #7533

Merged
wenshao merged 2 commits into
QwenLM:mainfrom
chinesepowered:fix/jsonl-empty-array-write
Jul 23, 2026
Merged

fix(core): jsonl write([]) should leave an empty file, not a stray newline#7533
wenshao merged 2 commits into
QwenLM:mainfrom
chinesepowered:fix/jsonl-empty-array-write

Conversation

@chinesepowered

Copy link
Copy Markdown
Contributor

What this PR does

jsonl.write(path, []) left a one-byte file containing a single \n. This terminates each record instead of joining with separators, so an empty array writes nothing. Output for a non-empty array is byte-identical.

Why it's needed

const lines = data.map((item) => JSON.stringify(item)).join('\n');
atomicWriteFileSync(filePath, `${lines}\n`, { encoding: 'utf8' });

For data = [] the join yields '', and the template literal then writes '\n'.

That makes the module's own accessors contradict each other:

after write(file, []) result
fs.statSync(file).size 1
exists(file) — documented as "exists and is not empty", tests size > 0 true
read(file) — skips blank lines []
countLines(file) — counts non-empty lines 0

So clearing a JSONL file leaves something that reports as non-empty while containing no records, and any caller that branches on exists() before reading takes the wrong path.

write is part of the public core API (export * from './utils/jsonl-utils.js' in packages/core/src/index.ts), so this is reachable by SDK and extension consumers even though no in-repo caller passes an empty array today. Fixing it now keeps a future write(file, []) from quietly hitting this.

Reviewer Test Plan

How to verify

  • From the repo root: npx vitest run --root packages/core src/utils/jsonl-utils.test.ts → 29/29.
  • The new test write() with an empty array leaves a genuinely empty file writes a record, clears the file with write(file, []), then asserts all four accessors agree: content '', size === 0, read() === [], exists() === false. Reverting only jsonl-utils.ts fails it with expected '\n' to be ''.
  • The existing write() full-file replaces existing content via atomic write and write() creates parent dirs when missing tests are untouched and still pass — they pin the non-empty output ('{"x":1}\n') that must not change.

Evidence (Before & After)

  • Before: write(f, []) → file is "\n", statSync().size === 1, exists(f) === true, read(f) === [].
  • After: write(f, []) → file is "", size === 0, exists(f) === false, read(f) === [].
  • Unchanged: write(f, [{v:10},{v:20}])'{"v":10}\n{"v":20}\n' in both cases.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

macOS: the jsonl-utils suite passes locally (29 tests) with proven fail-before/pass-after; eslint clean. String construction with no platform-dependent behavior — the test asserts on \n explicitly rather than on line endings the OS might vary — so no manual QA is required; CI covers Windows/Linux.

Environment (optional)

Node v24; @qwen-code/qwen-code-core workspace; vitest 3.2.

Risk & Scope

  • Main risk or tradeoff: none identified. The two forms produce identical bytes for every non-empty input (join('\n') + '\n' and per-item + '\n' differ only when the array is empty), which the untouched existing tests pin.
  • Not validated / out of scope: writeLine / writeLineSync are append helpers with no empty-input case and are unchanged.
  • Breaking changes / migration notes: none, unless something depended on a cleared JSONL file still reporting exists() === true — which is the bug being fixed.

Linked Issues

None — found by reading write against exists and read in the same module.

中文说明

本 PR 的作用

jsonl.write(path, []) 会留下一个仅含单个 \n 的 1 字节文件。本 PR 改为逐条记录追加终止符,而非用分隔符 join,使空数组写出空内容。非空数组的输出逐字节完全一致。

为什么需要

const lines = data.map((item) => JSON.stringify(item)).join('\n');
atomicWriteFileSync(filePath, `${lines}\n`, { encoding: 'utf8' });

data = [] 时 join 得到 '',模板字符串随即写入 '\n'

这导致该模块自身的各个访问函数互相矛盾:

执行 write(file, []) 结果
fs.statSync(file).size 1
exists(file)——文档写明「存在且非空」,判断 size > 0 true
read(file)——跳过空行 []
countLines(file)——统计非空行 0

于是清空一个 JSONL 文件后,会留下一个「报告为非空、却不含任何记录」的东西;任何在读取前依据 exists() 分支的调用方都会走错路径。

write 属于 core 的公开 API(packages/core/src/index.ts 中的 export * from './utils/jsonl-utils.js'),因此 SDK 与扩展使用方都可触达它——尽管当前仓库内尚无调用方传入空数组。现在修好,可避免将来某处 write(file, []) 悄悄踩中。

复核测试计划

如何验证

  • 在仓库根目录:npx vitest run --root packages/core src/utils/jsonl-utils.test.ts → 29/29 通过。
  • 新增测试 write() with an empty array leaves a genuinely empty file 先写入一条记录,再用 write(file, []) 清空,然后断言四个访问函数彼此一致:内容为 ''size === 0read() === []exists() === false。仅还原 jsonl-utils.ts 时该测试失败:expected '\n' to be ''
  • 既有的 write() full-file replaces existing content via atomic writewrite() creates parent dirs when missing 未作改动且仍然通过——它们固定了不得改变的非空输出('{"x":1}\n')。

证据(修复前后对比)

  • 修复前:write(f, []) → 文件为 "\n"statSync().size === 1exists(f) === trueread(f) === []
  • 修复后:write(f, []) → 文件为 ""size === 0exists(f) === falseread(f) === []
  • 不变:write(f, [{v:10},{v:20}]) → 两种实现均为 '{"v":10}\n{"v":20}\n'

测试环境

系统 状态
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

macOS:jsonl-utils 套件本地通过(29 个测试),并验证了 fail-before/pass-after;eslint 干净。纯字符串构造,无平台相关行为——测试显式断言 \n 而非依赖系统可能不同的行尾——因此无需人工 QA;Windows/Linux 由 CI 覆盖。

运行环境(可选)

Node v24;@qwen-code/qwen-code-core 工作区;vitest 3.2。

风险与影响范围

  • 主要风险或权衡:未发现。对任何非空输入,两种写法产生的字节完全相同(join('\n') + '\n' 与逐条 + '\n' 只在数组为空时才有差异),这一点由未改动的既有测试固定。
  • 未验证 / 范围之外:writeLine / writeLineSync 是追加型辅助函数,不存在空输入场景,未作改动。
  • 破坏性变更 / 迁移说明:无——除非有代码依赖「清空后的 JSONL 文件仍报告 exists() === true」,而那正是本次修复的 bug。

关联 Issue

无——通过在同一模块内对照阅读 writeexistsread 发现。

…wline

write() built its content with join('\n') and then appended a trailing
newline. For an empty array the join produces '', so the file ends up as
a single '\n': one byte, no records.

That makes the module's own accessors contradict each other — exists()
tests size > 0 and returns true, while read() skips blank lines and
returns []. Clearing a JSONL file therefore leaves something that reports
as non-empty but has nothing in it.

Terminate each record instead of joining with separators. The output for
a non-empty array is byte-identical; an empty array now writes nothing.
@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓

Problem: this is a real, self-evident inconsistency in the module's own API — write(file, []) produces a 1-byte "\n" file that exists() (size > 0) reports as non-empty while read() (skips blank lines) returns [] and countLines() returns 0. The PR documents the before/after clearly. No linked issue, but the bug is verifiable from the code alone.

Direction: aligned — a correctness fix in core JSONL utilities. No CHANGELOG reference needed for an internal consistency fix like this.

Size: core paths touched. Production lines: ~11 (jsonl-utils.ts), test lines: ~20 (jsonl-utils.test.ts). Well under any threshold.

Approach: the scope is exactly right — one function, one behavioral fix, one test. Switching from join('\n') + trailing \n to per-item \n termination + join('') is the simplest possible fix, and the output is byte-identical for non-empty arrays (the existing tests pin that). No unrelated changes, no scope creep.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓

问题:这是模块自身 API 中一个真实且显而易见的不一致——write(file, []) 会生成一个 1 字节的 "\n" 文件,exists()(size > 0)报告为非空,而 read()(跳过空行)返回 []countLines() 返回 0。PR 清晰地记录了修复前后的对比。虽然没有关联 issue,但这个 bug 仅从代码即可验证。

方向:对齐——核心 JSONL 工具的正确性修复。此类内部一致性修复无需 CHANGELOG 引用。

规模:触及核心路径。生产行数:约 11 行(jsonl-utils.ts),测试行数:约 20 行(jsonl-utils.test.ts)。远低于任何阈值。

方案:范围恰好——一个函数、一个行为修复、一个测试。从 join('\n') + 尾部 \n 改为逐条 \n 终止 + join('') 是最简方案,且非空数组的输出逐字节一致(既有测试固定了这一点)。无无关改动,无范围蔓延。

进入代码审查 🔍

Qwen Code · qwen3.8-max-preview

Reviewed at 58611ca4555c684cb3d3417d7d1e61a101cf8c99 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Code Review

Independent proposal: for write(file, []) leaving a stray \n, I'd switch from join('\n') + trailing \n to per-item \n termination + join('') — naturally produces '' for empty arrays, byte-identical for non-empty, no guard clause needed.

The PR takes exactly this approach. The diff is two lines of production code (plus a comment explaining the why) and one well-structured test. No correctness issues, no regressions, no convention violations. The comment in the production code is warranted — the why (empty-array edge case) is genuinely non-obvious from the code alone.

No downstream consumers of jsonl.write() exist in-repo (confirmed via grep), but it's public API via export * from './utils/jsonl-utils.js' in packages/core/src/index.ts, so the fix protects SDK/extension consumers.

Typecheck and ESLint both clean on the changed files.

Testing

Non-user-visible utility fix with no in-repo callers — unit tests are the appropriate verification (tmux not available on this runner).

With the fix (29/29 pass):

 RUN  v3.2.4 /home/github-runner/actions-runner-test-3/_work/qwen-code/qwen-code/packages/core

 ✓ src/utils/jsonl-utils.test.ts (29 tests) 29ms

 Test Files  1 passed (1)
      Tests  29 passed (29)
   Duration  3.45s

Without the fix (reverting only jsonl-utils.ts — the new test fails as expected):

 FAIL  src/utils/jsonl-utils.test.ts > writeLine / writeLineSync / write > write() with an empty array leaves a genuinely empty file
AssertionError: expected '\n' to be '' // Object.is equality

- Expected
+ Received

+
+

 ❯ src/utils/jsonl-utils.test.ts:370:43
    368|     write(file, []);
    369| 
    370|     expect(fs.readFileSync(file, 'utf8')).toBe('');
       |                                           ^
    371|     expect(fs.statSync(file).size).toBe(0);
    372|     expect(await read(file)).toEqual([]);

 Test Files  1 failed (1)
      Tests  1 failed | 28 passed (29)

Fail-before / pass-after confirmed — the test pins exactly the bug being fixed.

中文说明

代码审查

独立方案: 对于 write(file, []) 留下多余 \n 的问题,我会将 join('\n') + 尾部 \n 改为逐条 \n 终止 + join('')——空数组自然生成 '',非空数组逐字节一致,无需守卫子句。

PR 采用了完全相同的方案。diff 包含两行生产代码(加一条解释 why 的注释)和一个结构良好的测试。无正确性问题、无回归、无规范违反。生产代码中的注释是合理的——空数组边界情况从代码本身确实不易看出。

仓库内没有 jsonl.write() 的下游调用方(已通过 grep 确认),但它通过 packages/core/src/index.ts 中的 export * from './utils/jsonl-utils.js' 成为公开 API,因此此修复保护了 SDK/扩展使用方。

Typecheck 和 ESLint 在改动文件上均通过。

测试

非用户可见的工具函数修复,仓库内无调用方——单元测试是合适的验证方式(此运行环境无 tmux)。

修复后 29/29 通过;仅还原 jsonl-utils.ts 后新测试按预期失败:expected '\n' to be ''。fail-before / pass-after 已确认。

Qwen Code · qwen3.8-max-preview

Reviewed at 58611ca4555c684cb3d3417d7d1e61a101cf8c99 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 5/5 — clean across every stage; would merge without hesitation.

This is exactly what a good bugfix looks like: a real inconsistency in the module's own API, the simplest possible fix (two lines of production code), a comment that explains the non-obvious why, and a test that pins the behavior with proven fail-before/pass-after. The approach matches what I'd have done independently — per-item \n termination instead of join + append, byte-identical output for non-empty arrays, no guard clause needed.

No in-repo callers hit this today, but write is public API and the fix prevents a future write(file, []) from quietly producing a file that exists() and read() disagree about. Every change in the diff is necessary, nothing extraneous. Typecheck, lint, and all 29 tests green.

中文说明

置信度:5/5 —— 每个阶段都干净,毫不犹豫地合并。

这是一个优秀 bugfix 的典范:模块自身 API 中的真实不一致、最简方案(两行生产代码)、解释非显而易见 why 的注释、以及经 fail-before/pass-after 验证的行为固定测试。方案与我独立想到的完全一致——逐条 \n 终止替代 join + 追加,非空数组输出逐字节一致,无需守卫子句。

仓库内目前没有调用方触发此问题,但 write 是公开 API,此修复防止未来 write(file, []) 悄悄生成一个 exists()read() 互相矛盾的文件。diff 中每项改动都是必要的,无多余内容。Typecheck、lint、全部 29 个测试均通过。

Qwen Code · qwen3.8-max-preview

Reviewed at 58611ca4555c684cb3d3417d7d1e61a101cf8c99 · re-run with @qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Downgraded from Approve to Comment: CI failing: Test (ubuntu-latest, Node 22.x). Reviewed.

— qwen3.7-max via Qwen Code /review

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. LGTM! ✅

— qwen3.7-max via Qwen Code /review

@gwinthis

Copy link
Copy Markdown
Collaborator

Review & Local Verification Report

代码审查

设计评价:经典的 off-by-one 修复。 本 PR 修复了 write(file, []) 产生 1 字节文件(仅含 \n)的问题。

根因分析:

// Before: join + trailing newline
const lines = data.map(item => JSON.stringify(item)).join('\n');
atomicWriteFileSync(filePath, `${lines}\n`);
// [].join('\n') = '' → '' + '\n' = '\n' → 1-byte file

// After: terminate each record
const lines = data.map(item => `${JSON.stringify(item)}\n`).join('');
atomicWriteFileSync(filePath, lines);
// [] → '' → 0-byte file ✅
// [{a:1}] → '{"a":1}\n' ✅

影响: 修复前 exists()(检查 size > 0)与 read()(返回 [])对空文件语义不一致。修复后两者统一:空文件 = 0 字节 = 不存在。

测试: 验证 write(file, []) 后文件内容为 ''、大小为 0、read() 返回 []exists() 返回 false

结论

LGTM。 改动 3 行,语义正确,注释解释了 join vs terminate 的区别。

@doudouOUC doudouOUC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. LGTM! ✅

— qwen3.7-max via Qwen Code /review

@wenshao

wenshao commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

✅ Local Verification Report — PR #7533

Verified by: @wenshao (maintainer)
Date: 2026-07-23
Platform: macOS (darwin), Node v22.22.2
Branch: fix/jsonl-empty-array-write @ 05b1c6ae


Verification Steps

Step Command Result
Unit tests npx vitest run src/utils/jsonl-utils.test.ts ✅ 29/29 passed
TypeScript npx tsc --noEmit -p packages/core/tsconfig.json ✅ Clean
ESLint npx eslint packages/core/src/utils/jsonl-utils.{ts,test.ts} ✅ Clean
Manual E2E Custom script exercising write([]), write([{...}]), write([{...},{...}]) ✅ All 3 scenarios pass

Unit Test Results

Unit tests — 29/29 passed

Manual Verification — write([]) Behavior

The script calls the actual write(), read(), exists(), and countLines() from the patched module:

Manual verification — all accessors agree after write([])

Key observations from the manual run:

  • write(f, []) → file content "", size 0, exists()false, read()[], countLines()0. All four accessors are now consistent.
  • write(f, [{v:10},{v:20}])'{"v":10}\n{"v":20}\n' — byte-identical to the pre-fix output for non-empty arrays.
  • write(f, [{x:1}])'{"x":1}\n' — single-element case also unchanged.

Assessment

The fix is minimal, correct, and safe:

  1. Root causejoin('\n') + '\n' on an empty array produces '\n' (1 byte), which contradicts exists() (size > 0) vs read() (no records).
  2. Fix — terminate each record individually (map → JSON.stringify + '\n' → join('')), so an empty array yields ''.
  3. Non-empty output — byte-identical to the old implementation (verified by both the existing test and the manual script).
  4. New test — covers the exact regression: writes a record, clears with write([]), then asserts all four accessors agree.
  5. No regressions — all 29 existing + new tests pass; typecheck and lint clean.

Recommendation: ✅ Ready to merge.


🇨🇳 中文验证报告

✅ 本地验证报告 — PR #7533

验证人: @wenshao(维护者)
日期: 2026-07-23
平台: macOS (darwin),Node v22.22.2
分支: fix/jsonl-empty-array-write @ 05b1c6ae


验证步骤

步骤 命令 结果
单元测试 npx vitest run src/utils/jsonl-utils.test.ts ✅ 29/29 通过
TypeScript 类型检查 npx tsc --noEmit -p packages/core/tsconfig.json ✅ 无错误
ESLint npx eslint packages/core/src/utils/jsonl-utils.{ts,test.ts} ✅ 无警告
手动端到端验证 自定义脚本,分别测试 write([])write([{...}])write([{...},{...}]) ✅ 3 个场景全部通过

单元测试结果

单元测试 — 29/29 通过

手动验证 — write([]) 行为

脚本直接调用修复后模块的 write()read()exists()countLines()

手动验证 — write([]) 后所有访问器一致

手动运行的关键观察:

  • write(f, []) → 文件内容 "",大小 0exists()falseread()[]countLines()0。四个访问器现在完全一致。
  • write(f, [{v:10},{v:20}])'{"v":10}\n{"v":20}\n' — 与修复前非空数组的输出逐字节相同。
  • write(f, [{x:1}])'{"x":1}\n' — 单元素场景同样不变。

评估

修复最小、正确、安全:

  1. 根因 — 空数组经 join('\n') + '\n' 产生 '\n'(1 字节),导致 exists()(size > 0)与 read()(无记录)互相矛盾。
  2. 修复 — 改为逐条记录追加终止符(map → JSON.stringify + '\n' → join('')),空数组写出空字符串。
  3. 非空输出 — 与旧实现逐字节相同(既有测试和手动脚本均验证)。
  4. 新增测试 — 精确覆盖回归场景:先写一条记录,再用 write([]) 清空,然后断言四个访问器一致。
  5. 无回归 — 全部 29 个测试通过;类型检查和 lint 均干净。

建议:✅ 可以合入。

@wenshao
wenshao added this pull request to the merge queue Jul 23, 2026
@wenshao

wenshao commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Maintainer Local Verification Report

Verified locally on macOS (darwin 25.5.0, Node v22.22.2) against commit 58611ca on branch fix/jsonl-empty-array-write.

1. Unit tests — with fix (29/29 pass)

All 29 tests in jsonl-utils.test.ts pass, including the new write() with an empty array leaves a genuinely empty file test.

test-pass

2. Fail-before — old code + new test (1 failed)

Reverting only jsonl-utils.ts to origin/main while keeping the new test: the new test fails with expected '\n' to be '', confirming the test genuinely catches the bug.

test-fail-before

3. Behavioral verification — OLD vs NEW implementation

Ran a standalone script comparing the old (join('\n') + trailing \n) and new (per-item \n termination) implementations side by side:

Check Result
write(f, []) → OLD: "\n" (1 byte), NEW: "" (0 bytes) ✅ Fixed
Accessor consistency (size, exists(), read(), countLines()) ✅ All agree after fix
write(f, [{v:10},{v:20}]) byte-identical between OLD and NEW ✅ Yes
write(f, [{x:1}]) byte-identical between OLD and NEW ✅ Yes

behavioral-verify

Verdict

Ready to merge. The fix is minimal (2 lines of production code), correct, byte-identical for all non-empty inputs, and well-tested. CI is green. No downstream in-repo callers pass an empty array today, but the fix protects SDK/extension consumers of the public API.


中文说明

维护者本地验证报告

在 macOS(darwin 25.5.0, Node v22.22.2)上针对分支 fix/jsonl-empty-array-write 的提交 58611ca 进行了本地验证。

1. 单元测试——修复后(29/29 通过)

jsonl-utils.test.ts 中全部 29 个测试通过,包括新增的 write() with an empty array leaves a genuinely empty file 测试。

2. Fail-before——旧代码 + 新测试(1 个失败)

仅将 jsonl-utils.ts 还原到 origin/main,保留新测试:新测试以 expected '\n' to be '' 失败,确认该测试确实能捕获此 bug。

3. 行为验证——新旧实现对比

运行独立脚本,对比旧实现(join('\n') + 尾部 \n)和新实现(逐条 \n 终止):

检查项 结果
write(f, []) → 旧:"\n"(1 字节),新:""(0 字节) ✅ 已修复
访问器一致性(sizeexists()read()countLines() ✅ 修复后全部一致
write(f, [{v:10},{v:20}]) 新旧输出逐字节一致 ✅ 是
write(f, [{x:1}]) 新旧输出逐字节一致 ✅ 是

结论

可以合入。 修复极简(2 行生产代码),正确,对所有非空输入逐字节一致,测试充分。CI 全绿。当前仓库内无调用方传入空数组,但此修复保护了公开 API 的 SDK/扩展使用方。

— Maintainer local verification · macOS · Node v22.22.2 · 2026-07-23

Merged via the queue into QwenLM:main with commit baaedfb Jul 23, 2026
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants