Skip to content

test(desktop): settle the prompt rail across two consecutive reads #4685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Adarsh-Me wants to merge 3 commits into from
Closed

test(desktop): settle the prompt rail across two consecutive reads #4685

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
009fd2c
test(desktop): settle the prompt rail across painted frames (#4675)
Adarsh-Me Sep 3, 2026
d4c6677
test(desktop): gate the prompt rail on painted-frame quiescence (#4675)
Adarsh-Me Sep 3, 2026
ab2484d
test(desktop): pass the rail settle loop as a function, not source (#…
Adarsh-Me Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 112 additions & 63 deletions apps/desktop/e2e/prompt-rail.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,73 +137,122 @@ interface ActivePromptRailSnapshot {
sourceTurnId: string | null;
}

async function activePromptRailSnapshot(page: Page): Promise<ActivePromptRailSnapshot> {
return page.evaluate(async ({ promptCount }) => {
const root = document.querySelector<HTMLElement>('[data-chat-scroll-container="true"]');
if (!root) throw new Error('the chat scroll container is missing');
const ticks = [...document.querySelectorAll<HTMLElement>('.maka-prompt-rail-tick')];
const currentIds = ticks
.filter((tick) => tick.getAttribute('aria-current') === 'true')
.map((tick) => tick.dataset.promptTurnId ?? '');
const rootBounds = root.getBoundingClientRect();
const atEnd = root.scrollHeight - root.scrollTop - root.clientHeight <= 2;
const turns = [...root.querySelectorAll<HTMLElement>('[data-transcript-turn-id]')]
.map((turn) => ({
element: turn,
id: turn.dataset.transcriptTurnId ?? '',
index: Number(turn.dataset.transcriptTurnId?.split('-').at(-1)) - 1,
}))
.filter((turn) => turn.id.length > 0 && Number.isFinite(turn.index));
const readingBandTurns = turns
.filter(({ element }) => {
const bounds = element.getBoundingClientRect();
return bounds.bottom > rootBounds.top
&& bounds.top < rootBounds.top + rootBounds.height * 0.34;
})
.sort((left, right) => left.index - right.index);
const scrollportTurns = turns
.filter(({ element }) => {
const bounds = element.getBoundingClientRect();
return bounds.bottom > rootBounds.top && bounds.top < rootBounds.bottom;
})
.sort((left, right) => left.index - right.index);
const sourceTurn = atEnd
? turns.reduce<typeof turns[number] | null>(
(latest, turn) => latest === null || turn.index > latest.index ? turn : latest,
null,
)
: readingBandTurns[0] ?? scrollportTurns[0] ?? null;
const expectedRailIndex = sourceTurn === null || ticks.length === 0
? null
: Math.round(
sourceTurn.index * (ticks.length - 1) / (promptCount - 1),
);
const expectedId = expectedRailIndex === null
? null
: ticks[expectedRailIndex]?.dataset.promptTurnId ?? null;
return {
currentIds,
expectedId,
sourceTurnId: sourceTurn?.id ?? null,
};
}, { promptCount: PROMPT_RAIL_PROMPT_COUNT });
interface RailStateSnapshot extends ActivePromptRailSnapshot {
scrollTop: number;
scrollHeight: number;
clientHeight: number;
}

interface RailSettleOutcome {
settled: boolean;
state: RailStateSnapshot | null;
}

// One full rail read, in-page: the tick mapping plus the scroll metrics that
// feed it. A source string so the settle loop below and any diagnostic read
// run the exact same logic.
const READ_PROMPT_RAIL_STATE = `() => {
const promptCount = ${PROMPT_RAIL_PROMPT_COUNT};
const root = document.querySelector('[data-chat-scroll-container="true"]');
if (!root) throw new Error('the chat scroll container is missing');
const ticks = [...document.querySelectorAll('.maka-prompt-rail-tick')];
const currentIds = ticks
.filter((tick) => tick.getAttribute('aria-current') === 'true')
.map((tick) => tick.dataset.promptTurnId ?? '');
const rootBounds = root.getBoundingClientRect();
const atEnd = root.scrollHeight - root.scrollTop - root.clientHeight <= 2;
const turns = [...root.querySelectorAll('[data-transcript-turn-id]')]
.map((turn) => ({
element: turn,
id: turn.dataset.transcriptTurnId ?? '',
index: Number(turn.dataset.transcriptTurnId?.split('-').at(-1)) - 1,
}))
.filter((turn) => turn.id.length > 0 && Number.isFinite(turn.index));
const readingBandTurns = turns
.filter(({ element }) => {
const bounds = element.getBoundingClientRect();
return bounds.bottom > rootBounds.top
&& bounds.top < rootBounds.top + rootBounds.height * 0.34;
})
.sort((left, right) => left.index - right.index);
const scrollportTurns = turns
.filter(({ element }) => {
const bounds = element.getBoundingClientRect();
return bounds.bottom > rootBounds.top && bounds.top < rootBounds.bottom;
})
.sort((left, right) => left.index - right.index);
const sourceTurn = atEnd
? turns.reduce((latest, turn) => latest === null || turn.index > latest.index ? turn : latest, null)
: readingBandTurns[0] ?? scrollportTurns[0] ?? null;
const expectedRailIndex = sourceTurn === null || ticks.length === 0
? null
: Math.round(sourceTurn.index * (ticks.length - 1) / (promptCount - 1));
const expectedId = expectedRailIndex === null
? null
: ticks[expectedRailIndex]?.dataset.promptTurnId ?? null;
return {
currentIds,
expectedId,
sourceTurnId: sourceTurn?.id ?? null,
scrollTop: root.scrollTop,
scrollHeight: root.scrollHeight,
clientHeight: root.clientHeight,
};
}`;

// The rail resolves on the frame after a scroll and keeps re-resolving for
// six frames after each transcript mutation, and the transcript keeps
// remeasuring under them — so a snapshot can agree mid-settle and differ one
// mutation later. No fixed delay or pair of protocol reads proves the rail
// settled; that is how #4675 flaked and how frame-count fixes kept flaking
// under 4-worker stress. The observable quiet state is instead one read —
// mapping plus the scroll metrics that feed it — unchanged across six
// consecutive painted frames while mapping to the Turn being read. Every
// input the rail's resolver reacts to (scroll, mutation, geometry) is then
// exactly what produced the asserted state, so the reads a test makes after
// this helper see the same rail. The loop resolves with the state it
// verified; the passing path reads nothing after it.
const RAIL_QUIET_FRAMES = 6;
const RAIL_SETTLE_TIMEOUT_MS = 10_000;

const SETTLE_PROMPT_RAIL_SOURCE = `({ quietFrames, timeoutMs }) => new Promise((resolve) => {
const read = ${READ_PROMPT_RAIL_STATE};
let previousKey = null;
let quietFramesSeen = 0;
let lastState = null;
const timer = setTimeout(
() => resolve({ settled: false, state: lastState }),
timeoutMs,
);
const frame = () => {
const state = read();
const key = JSON.stringify(state);
quietFramesSeen = key === previousKey ? quietFramesSeen + 1 : 0;
previousKey = key;
lastState = state;
const agreed = state.expectedId !== null
&& state.currentIds.length === 1
&& state.currentIds[0] === state.expectedId;
if (agreed && quietFramesSeen >= quietFrames) {
clearTimeout(timer);
resolve({ settled: true, state });
return;
}
requestAnimationFrame(frame);
};
frame();
})`;

async function expectPromptRailMatchesReadingPosition(page: Page): Promise<void> {
let lastSnapshot: ActivePromptRailSnapshot | null = null;
try {
await expect.poll(async () => {
lastSnapshot = await activePromptRailSnapshot(page);
return lastSnapshot.expectedId !== null
&& lastSnapshot.currentIds.length === 1
&& lastSnapshot.currentIds[0] === lastSnapshot.expectedId;
}, { message: 'the one current tick maps from the Turn being read' }).toBe(true);
} catch {
throw new Error(`the prompt rail did not settle on the reading position: ${JSON.stringify(lastSnapshot)}`);
const outcome = await page.evaluate(SETTLE_PROMPT_RAIL_SOURCE, {

@hqhq1025 hqhq1025 Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Invoke the settle function instead of evaluating its source string

SETTLE_PROMPT_RAIL_SOURCE is a string, so Playwright sends it with isFunction: false: the expression produces an arrow function object, but Playwright does not call it or bind the second argument. The function result is not serializable and comes back as undefined, so the next line throws before any quiet-state frame is observed. This exact head fails the targeted 4-worker stress run 40/40 and the full spec fails 1/11 at this path. Pass an actual function callback to page.evaluate, or explicitly invoke the generated source with the serialized arguments, then rerun the same stress topology.

@Adarsh-Me Adarsh-Me Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and thank you for catching it — the finding is correct and the 40/40 result is consistent with the mechanism. I verified against playwright-core 1.62.1's injected evaluate: with isFunction: false the expression result is returned as-is (else if (isFunction === false) { result = result; }), so the generated arrow function was never invoked with the argument object and serialization reduced it to undefined; the helper then threw on outcome.settled before the first settle frame. The string-source form was an unjustified complication on my part — the old snapshot helper had already demonstrated the right pattern (a real function with the argument carrying promptCount).

Fixed on ab2484d: the settle loop is now passed to page.evaluate as a function, taking { promptCount, quietFrames, timeoutMs } — the same shape waitForStableTurnAtScrollerStart uses in transcript-scroll.spec.ts. The quiescence semantics from the previous commit are unchanged: full rail state (tick mapping plus scrollTop/scrollHeight/clientHeight) byte-identical across six consecutive painted frames while exactly one current tick maps from the Turn being read, the helper asserts the state the loop verified, and nothing is read after settle. The function-form also removes the duplication the string form created between source generation and the typed read.

Stress rerun request stands: --workers=4 --repeat-each=40 against ab2484d. This time the expected failure mode, if any remains, would be inside the helper's 10s in-page window rather than after it returns.

quietFrames: RAIL_QUIET_FRAMES,
timeoutMs: RAIL_SETTLE_TIMEOUT_MS,
}) as RailSettleOutcome;
if (!outcome.settled) {
throw new Error(`the prompt rail did not settle on the reading position: ${JSON.stringify(outcome.state)}`);
}
const snapshot = await activePromptRailSnapshot(page);
expect(snapshot.expectedId, `no visible Turn in ${JSON.stringify(snapshot)}`).not.toBeNull();
expect(snapshot.currentIds).toEqual([snapshot.expectedId]);
expect(outcome.state?.expectedId, `no visible Turn in ${JSON.stringify(outcome.state)}`).not.toBeNull();
expect(outcome.state?.currentIds).toEqual([outcome.state?.expectedId]);
}

async function scrollTranscriptThroughHistory(page: Page): Promise<void> {
Expand Down
Loading