Skip to content

fix: prevent diagnostics retention scroll lag - #703

Merged
Astro-Han merged 6 commits into
devfrom
codex/i700-diagnostics-retention
May 17, 2026
Merged

fix: prevent diagnostics retention scroll lag#703
Astro-Han merged 6 commits into
devfrom
codex/i700-diagnostics-retention

Conversation

@Astro-Han

@Astro-Han Astro-Han commented May 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • Rate-limit session.timeline.scroll_controller renderer diagnostics so scroll-heavy sessions cannot flood the log.
  • Trim renderer diagnostics to an 80% low-water mark instead of rewriting only back to the hard byte cap.
  • Split the diagnostics implementation into recorder, sanitizer, slice, and shared type modules, with focused tests for each layer.

Why

Installed builds could become scroll-laggy once renderer-diagnostics.jsonl sat near its 20 MiB cap. The trace did not show a renderer scroll/layout long task. Instead, clearing the diagnostics logs dropped INP from about 240ms to 16-40ms, and the log was dominated by session.timeline.scroll_controller events. The fix removes the noisy source and leaves headroom after retention so the app does not immediately re-enter the same rewrite path.

Related Issue

Closes #700

Human Review Status

Pending. A human should make the final merge decision after reviewing the final diff and verification evidence.

Review Focus

  • Whether session.timeline.scroll_controller belongs in the same high-frequency retention bucket as scroll and perf samples.
  • Whether the 80% low-water mark is the right smallest fix for avoiding repeated near-cap rewrites.
  • Whether the module split kept behavior unchanged outside the retention and rate-limit changes.

Risk Notes

Diagnostics may retain fewer older events when the file exceeds the cap because cleanup now targets 80% of the max. This is intentional; incident and session export behavior remains capped and sanitized. No user data migration, permissions, dependencies, or visible UI changes.

How To Verify

Diagnostics tests: 18 passed across renderer-diagnostics.test.ts, renderer-diagnostics-sanitize.test.ts, and renderer-diagnostics-slice.test.ts
Typecheck: bun run --cwd packages/desktop-electron typecheck passed with tsgo -b
Diff check: git diff --check passed with no whitespace errors

Screenshots or Recordings

Not required. This change is in desktop diagnostics retention and has no visible UI surface.

Checklist

  • Human review status is stated above as pending, approved, or not required
  • I linked the related issue, or stated why there is no issue
  • This PR has exactly one type label (bug, enhancement, task, or documentation), at least one primary routing label (app, ui, platform, harness, or ci), and exactly one priority label (P0 to P3), or I requested maintainer labeling
  • I described the review focus and any meaningful risks
  • I listed the relevant verification steps and the key result for each
  • I did not introduce unrelated refactors, dependencies, generated files, or file changes beyond the stated scope
  • I manually checked visible UI or copy changes when needed, with screenshots or recordings
  • I considered macOS and Windows impact for platform, packaging, updater, signing, paths, shell, or permissions changes
  • I called out docs, release notes, dependencies, permissions, credentials, deletion behavior, generated content, or local file changes when relevant
  • I reviewed the final diff for unrelated changes and suspicious dependency changes
  • I am targeting dev, and my PR title and commit messages use Conventional Commits in English

Summary by CodeRabbit

  • Refactoring

    • Reorganized renderer diagnostics system into modular components for improved maintainability and scalability.
  • New Features

    • Robust diagnostic event recording with configurable retention and byte-size limits.
    • Enhanced data sanitization with privacy and security controls.
    • Improved diagnostic log filtering, slicing, and export capabilities.
  • Tests

    • Expanded test coverage for recorder, sanitizer, and export functionality.

Review Change Stack

@Astro-Han Astro-Han added bug Something isn't working P1 High priority app Application behavior and product flows labels May 17, 2026
@coderabbitai

coderabbitai Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@Astro-Han has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 36 minutes and 36 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8c58fef4-b7f4-431d-9891-f45552a555e6

📥 Commits

Reviewing files that changed from the base of the PR and between e6192d8 and 4e37118.

📒 Files selected for processing (2)
  • packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts
  • packages/desktop-electron/src/main/renderer-diagnostics.test.ts
📝 Walkthrough

Walkthrough

This PR refactors and implements a complete renderer diagnostics pipeline, splitting a 639-line monolithic module into four focused submodules (types, sanitize, slice, recorder) with comprehensive tests, then re-exporting via a barrel. It adds event allowlisting, payload sanitization, byte-budget capping with retention headroom, serialized write queueing, and high-frequency rate-limiting to prevent diagnostic collection from degrading UI performance.

Changes

Renderer Diagnostics Capture and Retention

Layer / File(s) Summary
Types and configuration constants
packages/desktop-electron/src/main/renderer-diagnostics-types.ts
Defines RendererDiagnosticsStatus, event/input/export shapes, context/option types, and size/retention constants including low-watermark targets and high-frequency event thresholds.
Event sanitization and parsing
packages/desktop-electron/src/main/renderer-diagnostics-sanitize.ts
Implements allowlist-based event validation, string/numeric/boolean sanitizers with bounds checking, hostile pattern rejection (URLs, tokens), JSON byte sizing, and parseEventLine to construct sanitized RendererDiagnosticEvent objects.
Sanitizer test coverage
packages/desktop-electron/src/main/renderer-diagnostics-sanitize.test.ts
Tests validate allowlisted event acceptance, field filtering, disallowed field removal, URL/secret rejection, oversized payload rejection, and preservation of dotted identifiers.
Event filtering and byte capping
packages/desktop-electron/src/main/renderer-diagnostics-slice.ts
Implements event time parsing, session/incident classification, and selectRendererDiagnosticsSlice to filter by time and identity, cap by maxBytes with two-pass logic protecting incidents and identity transitions, and generate summary counts.
Slicing and export test coverage
packages/desktop-electron/src/main/renderer-diagnostics-slice.test.ts
Tests verify truncation status, session filtering, incident preservation, corrupt JSONL tolerance, byte-budget enforcement, and missing log status generation.
Recording, retention, and export
packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts
Implements exportRendererDiagnosticsLog to export/cap JSONL events, createRendererDiagnosticsRecorder with serialized write queueing, periodic retention flushing with temp-file safety and byte-target headroom, high-frequency event rate-limiting keyed by window/name, and writeFailed error tracking.
Recording and retention test coverage
packages/desktop-electron/src/main/renderer-diagnostics.test.ts
Tests verify event recording, high-frequency duplicate dropping for session.timeline.scroll_controller, retention headroom enforcement (staying under 80% capacity), and event report generation.
Module consolidation
packages/desktop-electron/src/main/renderer-diagnostics.ts
Transforms from monolithic implementation into barrel re-export of types, sanitize, slice, and recorder submodules for cohesive public API surface.

Sequence Diagram(s)

sequenceDiagram
  participant Input as RendererDiagnosticInput
  participant Sanitize as sanitizeRendererDiagnosticEvent
  participant Validate as validateEventName
  participant SizeCheck as jsonBytes
  participant Output as RendererDiagnosticEvent
  Input->>Sanitize: unknown input
  Sanitize->>Validate: check event name allowlist
  Validate-->>Sanitize: valid or undefined
  Sanitize->>SizeCheck: measure payload bytes
  SizeCheck-->>Sanitize: byte size
  Sanitize-->>Output: sanitized event or undefined
Loading
sequenceDiagram
  participant Client as Client Code
  participant Recorder as createRendererDiagnosticsRecorder
  participant Queue as enqueueWrite
  participant RateLimit as rateLimitByWindow
  participant Retention as maybeFlushRetention
  participant FS as Filesystem
  Client->>Recorder: record(diagnosticInput)
  Recorder->>Recorder: sanitizeRendererDiagnosticEvent
  Recorder->>RateLimit: check highFrequencyDrop
  RateLimit-->>Recorder: drop or proceed
  Recorder->>Queue: append to JSONL
  Queue->>FS: write appended event
  Recorder->>Retention: periodic maybeFlushRetention
  Retention->>FS: read existing JSONL
  Retention->>Retention: filter by age + cap by bytes
  Retention->>FS: write to temp, rename atomically
  FS-->>Retention: on rename error, cleanup
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Suggested labels

desktop

Poem

🐰 Logs now kept in trim and tidy rows,
Sanitized data, no secrets exposed,
Rate-limited scroll for smooth scrolling grace,
Headroom preserved in each diagnostic space!
Diagnose the UI without slowing its dance. 🎯

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: preventing diagnostics retention from causing scroll lag through rate-limiting and retention headroom adjustments.
Description check ✅ Passed The description comprehensively covers all required sections: summary of changes, rationale, related issue link, review focus, risk assessment, verification steps, and completed checklist items.
Linked Issues check ✅ Passed The PR implements all primary objectives from issue #700: rate-limits scroll_controller diagnostics [#700], adds 80% low-watermark retention headroom [#700], splits modules with focused tests [#700], and preserves export formats [#700].
Out of Scope Changes check ✅ Passed All changes align with stated objectives: module split, type definitions, sanitization logic, slice filtering, recorder implementation, and retention/rate-limiting. No unrelated refactors, dependencies, or generated files introduced.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/i700-diagnostics-retention

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added platform Electron shell, OS integration, packaging, updater, signing, paths, and permissions and removed app Application behavior and product flows labels May 17, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested priority: P2 (includes user-path files (packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts, packages/desktop-electron/src/main/renderer-diagnostics-sanitize.test.ts, packages/desktop-electron/src/main/renderer-diagnostics-sanitize.ts, packages/desktop-electron/src/main/renderer-diagnostics-slice.test.ts, packages/desktop-electron/src/main/renderer-diagnostics-slice.ts, packages/desktop-electron/src/main/renderer-diagnostics-types.ts, packages/desktop-electron/src/main/renderer-diagnostics.test.ts, packages/desktop-electron/src/main/renderer-diagnostics.ts)).

P1/P0 are reserved for maintainer confirmation. Please relabel manually if this is a release blocker, security issue, data-loss risk, or updater/runtime failure.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the renderer diagnostics system by modularizing the logic into separate files for recording, sanitization, slicing, and type definitions. The review feedback identifies performance bottlenecks in the retention and event capping logic, specifically pointing out that using Array.prototype.shift() and Array.prototype.splice() inside loops results in O(N^2) complexity. It is recommended to optimize these pruning operations to avoid potential main-process blocking when processing large diagnostic logs.

Comment thread packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts Outdated
Comment thread packages/desktop-electron/src/main/renderer-diagnostics-slice.ts Outdated
@Astro-Han
Astro-Han force-pushed the codex/i700-diagnostics-retention branch from 3e023f5 to 35a3e4d Compare May 17, 2026 14:40
@Astro-Han

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Astro-Han Astro-Han added P0 Blocking / highest priority and removed P1 High priority labels May 17, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts`:
- Around line 94-118: readEventReport currently returns a status ("ok" |
"missing" | "corrupt") but readEvents discards that and only returns events,
causing flushRetentionNow to treat a read error-as-corrupt as an empty event
list and potentially truncate the log; update the flow so flushRetentionNow
calls readEventReport (not readEvents) and checks the returned status,
immediately aborting/returning if status === "corrupt" (or handling "missing"
appropriately) before performing the temp-file rename; reference the functions
readEventReport, readEvents, and flushRetentionNow when making the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a81d26ab-f103-423a-8dc9-2264b83fbde5

📥 Commits

Reviewing files that changed from the base of the PR and between 3c1a6be and e6192d8.

📒 Files selected for processing (8)
  • packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts
  • packages/desktop-electron/src/main/renderer-diagnostics-sanitize.test.ts
  • packages/desktop-electron/src/main/renderer-diagnostics-sanitize.ts
  • packages/desktop-electron/src/main/renderer-diagnostics-slice.test.ts
  • packages/desktop-electron/src/main/renderer-diagnostics-slice.ts
  • packages/desktop-electron/src/main/renderer-diagnostics-types.ts
  • packages/desktop-electron/src/main/renderer-diagnostics.test.ts
  • packages/desktop-electron/src/main/renderer-diagnostics.ts

Comment thread packages/desktop-electron/src/main/renderer-diagnostics-recorder.ts
@Astro-Han
Astro-Han merged commit 1981523 into dev May 17, 2026
27 checks passed
@Astro-Han
Astro-Han deleted the codex/i700-diagnostics-retention branch May 17, 2026 16:03
Astro-Han added a commit that referenced this pull request May 17, 2026
Prepare PawWork v2026.5.18 for the stable desktop release.

- Bump the desktop package version to 2026.5.18.
- Scope the diagnostics unreadable-file retention test to POSIX permission semantics so Windows advisory does not fail on chmod behavior that Windows does not enforce the same way.

Verification:
- Focused desktop diagnostics test passed locally: 11 pass / 0 fail.
- Release typecheck passed locally for packages/desktop-electron.
- PR #706 CI passed, including ci, desktop-smoke, e2e-artifacts, CodeQL, dependency-review, label-policy, commit-lint, and title lint.

Release notes:
- Drafted against the merged range since v2026.5.17: #691, #692, #693, #694, #702, and #703.
- Cold-read review completed before merge; wording was tightened to avoid overclaiming diagnostics impact and to keep verification short.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working P0 Blocking / highest priority platform Electron shell, OS integration, packaging, updater, signing, paths, and permissions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Renderer diagnostics retention causes scroll input lag when full

1 participant