Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 118 additions & 104 deletions packages/core/src/services/backgroundShellRegistry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,32 @@ import {
type ShellTaskRegistration,
} from './backgroundShellRegistry.js';
import { todoWorkChainContext } from '../utils/promptIdContext.js';
import { escapeXml } from '../utils/xml.js';
import { stripDisplayControlChars } from '../utils/terminalSafe.js';

/**
* Builds the expected `<output-file>` element with the same
* `stripDisplayControlChars` + `escapeXml` pipeline the registry applies.
* Expected paths below come from `tmpdir()`, which can legally contain XML
* metacharacters (`&` on Windows, `<` on POSIX) or bidi overrides, so
* hand-rolling the escaping would make these cases depend on the host's TMPDIR.
*/
function expectedOutputFileElement(path: string): string {
return `<output-file>${escapeXml(stripDisplayControlChars(path))}</output-file>`;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Keep this expected sanitizer aligned with the registry implementation. The imported terminalSafe.ts helper is not the helper used by backgroundShellRegistry.ts: this one also strips U+202A–U+202E and U+2066–U+2069, while the registry's private same-named function only strips C0/C1 characters. A legal POSIX TMPDIR containing one of those Unicode characters therefore makes the expected path lose a character while the emitted XML retains it. I reproduced this at the current head with TMPDIR=.../tmp-\u202e-marker; both emits one task-notification when a shell completes and the control-character notification test fail. Since this helper was added specifically to avoid host-dependent expectations, please either calculate the expected value with the registry's actual semantics or make the registry deliberately reuse this shared sanitizer (and cover that behavior).

}

let tmpDirs: string[] = [];
let tmpFiles: string[] = [];

afterEach(() => {
for (const dir of tmpDirs) {
rmSync(dir, { recursive: true, force: true });
}
for (const file of tmpFiles) {
rmSync(file, { force: true });
}
tmpDirs = [];
tmpFiles = [];
});

function makeOutputFile(content: string): string {
Expand All @@ -54,19 +72,34 @@ function makeTempDir(): string {
function makeEntry(
overrides: Partial<ShellTaskRegistration> = {},
): ShellTaskRegistration {
const shellId = overrides.shellId ?? 's1';
return {
shellId: 's1',
shellId,
command: 'sleep 60',
cwd: '/tmp',
status: 'running',
startTime: 1000,
outputPath: '/tmp/s1.output',
abortController: new AbortController(),
...overrides,
// Every register/complete/fail/cancel mirrors the entry into a
// `<outputPath>.status` sidecar, so the default outputPath decides where
// that write lands. A fixed `/tmp/s1.output` pointed every entry in this
// file — across tests, across workers, across CI jobs — at the single
// path `/tmp/s1.status`. `/tmp` is sticky, so once that file belongs to
// another uid the atomic rename fails EPERM, and `renameWithRetrySync`
// burns its full 50+100+200ms backoff before the registry swallows the
// error. Give each entry its own directory instead: no shared state, and
// the sidecar write actually succeeds.
outputPath:
overrides.outputPath ?? join(makeTempDir(), `shell-${shellId}.output`),
Comment on lines +93 to +94

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The pre-existing makeDirEntry helper (~lines 782-795) now duplicates this construction: it independently builds the same makeTempDir() + shell-<shellId>.output shape with the same ?? 's1' default, plus hand-derives statusPath: join(dir, \shell-${shellId}.status`)— which the already-importedstatusFilePathForproduces directly. — Concrete cost: a future change to the canonicalshell-.output` naming scheme must be made in two places in this file, and the sidecar suite would silently keep exercising the old shape (its explicit override wins over the updated default) while every other test uses the new one.

function makeDirEntry(
  overrides: Partial<ShellTaskRegistration> = {},
): ShellTaskRegistration & { statusPath: string } {
  const entry = makeEntry(overrides);
  return { ...entry, statusPath: statusFilePathFor(entry.outputPath) };
}
中文说明

已有的 makeDirEntry 辅助函数(约 L782-795)现在与这段构造重复:它独立构造了同样的 makeTempDir() + shell-<shellId>.output 路径形状和同样的 ?? 's1' 默认值,还手工推导 statusPath: join(dir, \shell-${shellId}.status`)——而这正是已导入的 statusFilePathFor能直接给出的。— 具体代价:将来若修改shell-.output` 的规范命名,需要在本文件两处同步修改;而且 sidecar 测试组会因其显式覆盖优先于新默认值而悄悄继续验证旧命名,其余测试却已用上新命名。

— qwen3.8-max via Qwen Code /review (v0.21.8)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in cc588b2. makeDirEntry is now just makeEntry(overrides) plus statusFilePathFor(entry.outputPath), so the naming scheme lives in one place and the sidecar suite exercises the same shape as everything else.

Worth noting the side effect: because that group now inherits the default outputPath instead of overriding it, reverting makeEntry to the old constant fails four tests rather than one — the uniqueness guard plus three sidecar cases. The dedupe bought a stronger regression net than it looks.

Comment on lines +93 to +94

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Two sibling tests in this file still override outputPath with fixed /tmp paths — '/tmp/out&err.log' (~line 337, register + fail) and '/tmp/out\x03.log' (~line 409, register + complete) — keeping exactly the shared-sidecar pattern this change removes from the default; each performs two sidecar writes to a fixed /tmp/<name>.status path. — Failure scenario: probe-measured on this runner: with a root-owned squatter file planted at the path in sticky /tmp, these two tests went from 72 ms combined to 705 ms / 702 ms (2 writes × the 50+100+200 ms EPERM backoff the new comment documents), and two stray .status files leak into /tmp per run because afterEach only removes the mkdtempSync dirs.

outputPath: join(makeTempDir(), 'out&err.log'), // and likewise for the \x03 sibling

(relaxing the two <output-file> XML assertions to match the escaped suffix, e.g. toContain('out&amp;err.log')).

中文说明

本文件里另有两个兄弟测试仍然用固定的 /tmp 路径覆盖 outputPath——'/tmp/out&err.log'(约 L337,register + fail)和 '/tmp/out\x03.log'(约 L409,register + complete)——保留了本次改动刚从默认值中消除的同一种共享 sidecar 模式;它们各自向固定的 /tmp/<name>.status 路径写两次 sidecar。— 故障场景:在本 runner 上实测:在带 sticky 位的 /tmp 中放置一个 root 所有的占位文件后,这两个测试从合计 72 ms 变为 705 ms / 702 ms(2 次写 × 新注释所记载的 50+100+200 ms EPERM 退避),并且每次运行都会向 /tmp 泄漏两个残留 .status 文件,因为 afterEach 只清理 mkdtempSync 目录。

— qwen3.8-max via Qwen Code /review (v0.21.8)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in cc588b2 — both siblings now use join(makeTempDir(), ...). Confirmed the leak you described is gone: rm -f /tmp/*.status, run the suite, and no .status file appears in /tmp afterwards.

One follow-up in 9158922: moving to random temp dirs had relaxed the two <output-file> assertions to a suffix match. Rebuilt the full element from the path under test instead (<output-file>${outputPath.replaceAll("&", "&amp;")}</output-file>, and join(dir, "out.log") for the control-byte sibling) — the prefix is random but the escaping and stripping these cases pin are exact, so they should stay anchored.

Comment on lines +93 to +94

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The default-outputPath uniqueness this PR exists to guarantee is not pinned by any assertion. — Failure scenario: probe-measured: reverting makeEntry to the pre-PR constant '/tmp/s1.output' passed all 57 existing tests — only the guard proposed below failed. On a single-uid machine or fresh CI container that regression is invisible; on a shared multi-tenant runner the collision flake this PR fixes would return, with no in-repo guard pointing at the cause.

it('gives each entry a unique default outputPath', () => {
  expect(makeEntry().outputPath).not.toBe(makeEntry().outputPath);
});
中文说明

本 PR 要保证的默认 outputPath 唯一性,目前没有任何断言锁定。— 故障场景:实测将 makeEntry 还原为 PR 前的常量 '/tmp/s1.output' 后,现有 57 个测试全部通过——只有下面建议的守卫测试会失败。在单 uid 机器或全新 CI 容器上,这种回归完全无感;而在多租户共享 runner 上,本 PR 修复的冲突 flake 会再次出现,且仓库里没有任何守卫能指向成因。

— qwen3.8-max via Qwen Code /review (v0.21.8)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair hit — this was the gap in my verification. I A/B tested the cost (128.9s → 3.2s with the path made immutable) but never checked that reverting the change fails anything, and you are right that it did not.

Guard added in cc588b2. Mutation-checked: restoring outputPath: "/tmp/s1.output" now fails four tests (this guard plus three in the sidecar group, which picks up the default after the makeDirEntry dedupe), where before it failed none.

};
}

describe('BackgroundShellRegistry', () => {
it('gives each entry a unique default outputPath', () => {
expect(makeEntry().outputPath).not.toBe(makeEntry().outputPath);
});

describe('register / get / getAll', () => {
it('captures the Todo work-chain owner at registration', () => {
const reg = new BackgroundShellRegistry();
Expand Down Expand Up @@ -282,7 +315,7 @@ describe('BackgroundShellRegistry', () => {
expect(modelText).toContain(
'<output-tail truncated="false">first line\nfinal result</output-tail>',
);
expect(modelText).toContain(`<output-file>${outputPath}</output-file>`);
expect(modelText).toContain(expectedOutputFileElement(outputPath));
expect(meta).toEqual({
shellId: 'a',
status: 'completed',
Expand Down Expand Up @@ -318,12 +351,13 @@ describe('BackgroundShellRegistry', () => {
const reg = new BackgroundShellRegistry();
const callback = vi.fn();
reg.setNotificationCallback(callback);
const outputPath = join(makeTempDir(), 'out&err.log');
reg.register(
makeEntry({
shellId: 'a&b',
command: 'echo "<script>"',
cwd: '/repo&work',
outputPath: '/tmp/out&err.log',
outputPath,
}),
);

Expand All @@ -337,9 +371,9 @@ describe('BackgroundShellRegistry', () => {
);
expect(modelText).toContain('<cwd>/repo&amp;work</cwd>');
expect(modelText).toContain('<result>bad &lt;thing&gt;[31m</result>');
expect(modelText).toContain(
'<output-file>/tmp/out&amp;err.log</output-file>',
);
// Assert the whole element, not just the tail: the temp prefix is
// random but the escaping is what this test is about.
expect(modelText).toContain(expectedOutputFileElement(outputPath));
});

it('limits output-tail to the retained byte budget', () => {
Expand Down Expand Up @@ -387,26 +421,32 @@ describe('BackgroundShellRegistry', () => {
expect(modelText).not.toContain('\uFFFD');
});

it('strips control characters from cwd and output-file XML fields', () => {
it('strips control and bidi characters from cwd and output-file XML fields', () => {
const reg = new BackgroundShellRegistry();
const callback = vi.fn();
reg.setNotificationCallback(callback);
const dir = makeTempDir();
reg.register(
makeEntry({
shellId: 'a',
cwd: '/repo\x01\x02/work',
outputPath: '/tmp/out\x03.log',
outputPath: join(dir, 'out\x03\u202e.log'),
}),
);

reg.complete('a', 0, 2000);

const [, modelText] = callback.mock.calls[0];
expect(modelText).toContain('<cwd>/repo/work</cwd>');
expect(modelText).toContain('<output-file>/tmp/out.log</output-file>');
// Whole element: pins exactly which characters are stripped and that
// the rest of the path survives intact.
expect(modelText).toContain(
expectedOutputFileElement(join(dir, 'out.log')),
);
expect(modelText).not.toContain('\x01');
expect(modelText).not.toContain('\x02');
expect(modelText).not.toContain('\x03');
expect(modelText).not.toContain('\u202e');
});

const itNoFollow = fsConstants.O_NOFOLLOW === undefined ? it.skip : it;
Expand Down Expand Up @@ -457,6 +497,10 @@ describe('BackgroundShellRegistry', () => {
const reg = new BackgroundShellRegistry();
const callback = vi.fn();
const dir = makeTempDir();
// A dir outputPath gets its `<dir>.status` sidecar as a sibling of
// the temp dir, which the dir cleanup above never removes; tracking
// it here lets afterEach delete it even if the assertions fail.
tmpFiles.push(statusFilePathFor(dir));
reg.setNotificationCallback(callback);
reg.register(makeEntry({ shellId: 'a', outputPath: dir }));

Expand Down Expand Up @@ -669,95 +713,73 @@ describe('BackgroundShellRegistry', () => {
});
});

// Every register/complete in these loop tests also writes the status
// sidecar through atomicWriteFileSync, and a loaded CI runner has been
// measured spending ~700ms per sidecar write — ~50s for the ~70 writes
// of the longest loop, past the 15s default. The explicit timeout buys
// the I/O the time it costs; the assertions are unchanged.
const SIDECAR_IO_TIMEOUT = 120_000;
describe('terminal-entry retention cap', () => {
it(
'retains only a bounded number of terminal entries (oldest by endTime evicted)',
() => {
const reg = new BackgroundShellRegistry();
// Register and complete one more entry than the cap allows. Use
// strictly increasing endTimes so eviction order is deterministic.
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS + 2; i++) {
reg.register(makeEntry({ shellId: `s-${i}`, startTime: i * 10 }));
reg.complete(`s-${i}`, 0, i * 10 + 5);
}
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
// The two oldest (`s-0`, `s-1`) get pruned; the newest survives.
expect(reg.get('s-0')).toBeUndefined();
expect(reg.get('s-1')).toBeUndefined();
expect(reg.get(`s-${MAX_RETAINED_TERMINAL_SHELLS + 1}`)).toBeDefined();
},
SIDECAR_IO_TIMEOUT,
);

it(
'never evicts running entries even when the cap is exceeded',
() => {
const reg = new BackgroundShellRegistry();
// Register one extra terminal entry beyond the cap, then a single
// running entry. The running entry must be retained regardless of
// its launch order — pruning a still-running shell would lose the
// user's only handle on a live process.
reg.register(makeEntry({ shellId: 'live', startTime: 1 }));
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS + 1; i++) {
reg.register(
makeEntry({ shellId: `done-${i}`, startTime: 100 + i * 10 }),
);
reg.complete(`done-${i}`, 0, 100 + i * 10 + 5);
}
// Cap-of-32 terminals + 1 running survivor = 33 entries kept.
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS + 1);
expect(reg.get('live')?.status).toBe('running');
// The oldest terminal entry (lowest endTime) is the one evicted.
expect(reg.get('done-0')).toBeUndefined();
},
SIDECAR_IO_TIMEOUT,
);
it('retains only a bounded number of terminal entries (oldest by endTime evicted)', () => {
const reg = new BackgroundShellRegistry();
// Register and complete one more entry than the cap allows. Use
// strictly increasing endTimes so eviction order is deterministic.
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS + 2; i++) {
reg.register(makeEntry({ shellId: `s-${i}`, startTime: i * 10 }));
reg.complete(`s-${i}`, 0, i * 10 + 5);
}
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
// The two oldest (`s-0`, `s-1`) get pruned; the newest survives.
expect(reg.get('s-0')).toBeUndefined();
expect(reg.get('s-1')).toBeUndefined();
expect(reg.get(`s-${MAX_RETAINED_TERMINAL_SHELLS + 1}`)).toBeDefined();
});

it(
'prunes after fail() too, not just complete()',
() => {
const reg = new BackgroundShellRegistry();
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS; i++) {
reg.register(makeEntry({ shellId: `done-${i}`, startTime: i * 10 }));
reg.complete(`done-${i}`, 0, i * 10 + 5);
}
const overflowStart = MAX_RETAINED_TERMINAL_SHELLS * 10 + 100;
it('never evicts running entries even when the cap is exceeded', () => {
const reg = new BackgroundShellRegistry();
// Register one extra terminal entry beyond the cap, then a single
// running entry. The running entry must be retained regardless of
// its launch order — pruning a still-running shell would lose the
// user's only handle on a live process.
reg.register(makeEntry({ shellId: 'live', startTime: 1 }));
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS + 1; i++) {
reg.register(
makeEntry({ shellId: 'overflow', startTime: overflowStart }),
makeEntry({ shellId: `done-${i}`, startTime: 100 + i * 10 }),
);
reg.fail('overflow', 'boom', overflowStart + 5);
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
expect(reg.get('done-0')).toBeUndefined();
expect(reg.get('overflow')?.status).toBe('failed');
},
SIDECAR_IO_TIMEOUT,
);
reg.complete(`done-${i}`, 0, 100 + i * 10 + 5);
}
// Cap-of-32 terminals + 1 running survivor = 33 entries kept.
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS + 1);
expect(reg.get('live')?.status).toBe('running');
// The oldest terminal entry (lowest endTime) is the one evicted.
expect(reg.get('done-0')).toBeUndefined();
});

it(
'prunes after cancel() too, not just complete()',
() => {
const reg = new BackgroundShellRegistry();
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS; i++) {
reg.register(makeEntry({ shellId: `done-${i}`, startTime: i * 10 }));
reg.complete(`done-${i}`, 0, i * 10 + 5);
}
const overflowStart = MAX_RETAINED_TERMINAL_SHELLS * 10 + 100;
reg.register(
makeEntry({ shellId: 'overflow', startTime: overflowStart }),
);
reg.cancel('overflow', overflowStart + 5);
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
expect(reg.get('done-0')).toBeUndefined();
expect(reg.get('overflow')?.status).toBe('cancelled');
},
SIDECAR_IO_TIMEOUT,
);
it('prunes after fail() too, not just complete()', () => {
const reg = new BackgroundShellRegistry();
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS; i++) {
reg.register(makeEntry({ shellId: `done-${i}`, startTime: i * 10 }));
reg.complete(`done-${i}`, 0, i * 10 + 5);
}
const overflowStart = MAX_RETAINED_TERMINAL_SHELLS * 10 + 100;
reg.register(
makeEntry({ shellId: 'overflow', startTime: overflowStart }),
);
reg.fail('overflow', 'boom', overflowStart + 5);
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
expect(reg.get('done-0')).toBeUndefined();
expect(reg.get('overflow')?.status).toBe('failed');
});

it('prunes after cancel() too, not just complete()', () => {
const reg = new BackgroundShellRegistry();
for (let i = 0; i < MAX_RETAINED_TERMINAL_SHELLS; i++) {
reg.register(makeEntry({ shellId: `done-${i}`, startTime: i * 10 }));
reg.complete(`done-${i}`, 0, i * 10 + 5);
}
const overflowStart = MAX_RETAINED_TERMINAL_SHELLS * 10 + 100;
reg.register(
makeEntry({ shellId: 'overflow', startTime: overflowStart }),
);
reg.cancel('overflow', overflowStart + 5);
expect(reg.getAll()).toHaveLength(MAX_RETAINED_TERMINAL_SHELLS);
expect(reg.get('done-0')).toBeUndefined();
expect(reg.get('overflow')?.status).toBe('cancelled');
});
});

describe('cancel', () => {
Expand Down Expand Up @@ -793,16 +815,8 @@ describe('BackgroundShellRegistry', () => {
function makeDirEntry(
overrides: Partial<ShellTaskRegistration> = {},
): ShellTaskRegistration & { statusPath: string } {
const dir = makeTempDir();
const shellId = (overrides.shellId as string) ?? 's1';
const entry = makeEntry({
outputPath: join(dir, `shell-${shellId}.output`),
...overrides,
});
return {
...entry,
statusPath: join(dir, `shell-${shellId}.status`),
};
const entry = makeEntry(overrides);
return { ...entry, statusPath: statusFilePathFor(entry.outputPath) };
}

function readStatus(statusPath: string): Record<string, unknown> {
Expand Down
22 changes: 1 addition & 21 deletions packages/core/src/services/backgroundShellRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,34 +24,14 @@ import type { TaskBase, TaskRegistration } from '../agents/tasks/types.js';
import { atomicWriteFileSync } from '../utils/atomicFileWrite.js';
import { createDebugLogger } from '../utils/debugLogger.js';
import { todoWorkChainContext } from '../utils/promptIdContext.js';
import { stripDisplayControlChars } from '../utils/terminalSafe.js';
import { escapeXml } from '../utils/xml.js';

const debugLogger = createDebugLogger('BACKGROUND_SHELLS');
const MAX_NOTIFICATION_COMMAND_LENGTH = 80;
const MAX_NOTIFICATION_MODEL_COMMAND_LENGTH = 500;
export const MAX_NOTIFICATION_OUTPUT_TAIL_BYTES = 8192;

/**
* Strip C0 control characters (except tab) and C1 control characters from
* terminal/UI display strings. Shell commands and errors are usually
* user-authored, but this keeps escape sequences out of the visible
* notification surface if a caller passes unsanitized text.
*/
function stripDisplayControlChars(text: string): string {
let out = '';
for (let i = 0; i < text.length; i++) {
const code = text.charCodeAt(i);
if (code === 0x09) {
out += text[i];
continue;
}
if (code < 0x20) continue;
if (code >= 0x80 && code <= 0x9f) continue;
out += text[i];
}
return out;
}

function stripOutputControlChars(text: string): string {
let out = '';
for (let i = 0; i < text.length; i++) {
Expand Down
Loading