fix(logs): declutter and clarify the readable session log - #21
Merged
Conversation
- strip_ansi removes raw escape sequences from tool output before truncation, so colored subprocess output (e.g. vitest) no longer shows up as garbled control-byte text in the readable .log - format-stream.jq now walks the stream statefully via foreach, correlating tool_result events back to their tool_use by id - Read tool_result lines are dropped entirely: the paired '> Read(...)' line already shows file path and offset/limit, so the truncated file-content dump was pure redundant noise - overnight.sh appends genuine tool/harness errors (is_error results - permission denials, bad exit codes, missing files) to the same errlog used for the claude process's own stderr, then deletes the file if nothing landed in it Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Glob/Grep results collapse to a one-line count summary instead of dumping every matched path/line - Bash results get tool-aware summarization: vitest's Test Files/Tests lines plus each FAILED test, tsc's error lines plus the closing Found-N-errors line, eslint's problems re-attached to their file, and a last-line collapse for bracketed-status tool output (e.g. codegen's [STARTED]/[SUCCESS] chatter) - pure file-inspection Bash chains (cat/ls/echo/head/tail joined by &&/;/||, e.g. "cat a && echo --- && cat b") are suppressed entirely, same rationale as the existing Read suppression - the request line already shows what was inspected - strip_cr removes stray \r bytes so Windows-line-ended file dumps don't render as phantom blank lines - indent_continuations aligns multi-line result bodies under the " < " prefix instead of falling back to column 0 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- add find to the pure-inspect verb group so fallback chains like "cat a.ts || cat a.yml || find . -iname ..." get suppressed - add a second inspect group for tasklist/ps (with an allowed "| grep" suffix) so process-listing checks don't dump PIDs/memory - loosen the bracket-tag collapse: real tool output often has preamble before the [STARTED]/[SUCCESS] lines, so require only 2+ tagged lines plus a completion-looking last line instead of nearly every line being tagged - Bash-invoked "grep ... " calls (optionally behind "cd ... &&" and a trailing "| head/tail") now get the same one-line match-count summary as the dedicated Grep tool instead of a raw line dump Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- git status --short/-s condenses to a per-status-code count (e.g. "15 modified, 2 untracked") instead of truncating to the last 5 files, which was silently dropping real changes on anything larger - git commit output drops the lefthook/commitlint box-drawing noise and per-file create/delete-mode listing, keeping just the "[branch hash] subject" line and the diffstat - the request line for a "git commit -m ..." call now shows just the commit title instead of raw-truncating the JSON-embedded multi-line message mid-body - git's core.autocrlf "LF will be replaced by CRLF" warnings are stripped everywhere - never actionable - git diff output is dropped entirely (redundant with the PR itself) - Bash-run dev server startup collapses to just the "Ready in Xs" line when present; startup failures still fall through to the existing tail-5 fallback Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe formatter now processes NDJSON streams with stateful tool-use correlation, cleaned tool output, tool-specific summaries, inspection suppression, and truncation. ChangesStream formatting and error capture
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Claude as Claude stream
participant Formatter as format-stream.jq
participant Results as Tool results
participant Overnight as overnight.sh
Claude->>Formatter: Emit NDJSON events
Formatter->>Formatter: Store tool-use IDs
Results-->>Formatter: Provide scalar or array content
Formatter->>Formatter: Clean and summarize correlated output
Formatter-->>Claude: Emit formatted events
Claude->>Overnight: Complete stream and raw_log
Overnight->>Overnight: Extract, clean, and append tool errors
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@format-stream.jq`:
- Around line 199-214: Update the tool-result branching in format-stream.jq to
handle .is_error == true before the Read and pure-inspection Bash suppression
branches. Write failed correlated inspection results through to the live
readable output, while preserving suppression for successful Read and
pure-inspection Bash results and existing formatting for Glob and Grep.
- Around line 11-12: Update strip_ansi in format-stream.jq (format-stream.jq
lines 11-12) and the matching error handler in overnight.sh (overnight.sh line
130) to use the complete CSI escape-sequence pattern, including optional
private-mode markers such as ?. Ensure both log paths remove sequences like
ESC[?25l.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 46e4221b-4bdf-4604-bfff-19ab3d0ed37c
📒 Files selected for processing (2)
format-stream.jqovernight.sh
Comment on lines
+11
to
+12
| def strip_ansi: | ||
| gsub("\\[[0-9;]*[a-zA-Z]"; ""); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sample=$'\033[?25lhidden\033[0m'
printf '%s' "$sample" |
jq -Rr 'gsub("\u001b\\[[0-9;]*[a-zA-Z]"; "")' |
od -An -tx1
printf '%s' "$sample" |
jq -Rr 'gsub("\u001b\\[[0-?]*[ -/]*[`@-`~]"; "")' |
od -An -tx1Repository: c0d3ster/nightlight
Length of output: 197
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Files:\n'
git ls-files | rg '(^|/)(format-stream\.jq|overnight\.sh)$' || true
printf '\nRelevant snippets:\n'
if [ -f format-stream.jq ]; then
nl -ba format-stream.jq | sed -n '1,25p'
fi
if [ -f overnight.sh ]; then
nl -ba overnight.sh | sed -n '120,140p'
fi
printf '\nPattern occurrences:\n'
rg -n 'gsub\("\u001b|\[0-9;\]\*|\[0-?]\*\|private|CSI|strip_ansi' format-stream.jq overnight.sh || true
printf '\nBehavioral probe with Python (no repo code/exec):\n'
python3 - <<'PY'
import re
samples = [
"\033[?25lhidden\033[0m",
"\033[?1049h\033[2J\033[H",
"\033[38;2;255;128;0mcolored",
]
patterns = {
"current": re.compile(r"\x1b\[[0-9;]*[a-zA-Z]"),
"complete": re.compile(r"\x1b\[[0-?]*[ -/]*[`@-`~]"),
}
for sample in samples:
print(f"INPUT={sample!r}")
for name, re_pat in patterns.items():
print(f" {name}={re_pat.sub('', sample)!r}")
PYRepository: c0d3ster/nightlight
Length of output: 254
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Relevant snippets:\n'
if [ -f format-stream.jq ]; then
line=1
while IFS= read -r content || [ -n "$content" ]; do
printf '%d:%s\n' "$line" "$content"
[ "$line" -ge 25 ] && break
line=$((line + 1))
done < format-stream.jq
fi
if [ -f overnight.sh ]; then
line=1
while IFS= read -r content || [ -n "$content" ]; do
printf '%d:%s\n' "$line" "$content"
[ "$line" -ge 140 ] && break
line=$((line + 1))
done < overnight.sh
fi
printf '\nPattern occurrences:\n'
rg -n 'gsub\("\u001b|\[0-9;\]\*|\[0-?]\*\|private|CSI|strip_ansi' format-stream.jq overnight.sh nightly.sh .github 2>/dev/null || true
printf '\nBehavioral probe with Python:\n'
python3 - <<'PY'
import re
samples = [
"\033[?25lhidden\033[0m",
"\033[?1049h\033[2J\033[H",
"\033[38;2;255;128;0mcolored",
]
patterns = {
"current": re.compile(r"\x1b\[[0-9;]*[a-zA-Z]"),
"complete": re.compile(r"\x1b\[[0-?]*[ -/]*[`@-`~]"),
}
for sample in samples:
print(f"INPUT={sample!r}")
for name, regex in patterns.items():
print(f" {name}={regex.sub('', sample)!r}")
PYRepository: c0d3ster/nightlight
Length of output: 8535
Remove private CSI escape sequences in both log paths.
Use the complete CSI regex in format-stream.jq and the matching error handler in overnight.sh so private sequences such as ESC[?25l do not remain in the logs.
📍 Affects 2 files
format-stream.jq#L11-L12(this comment)overnight.sh#L130-L130
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@format-stream.jq` around lines 11 - 12, Update strip_ansi in format-stream.jq
(format-stream.jq lines 11-12) and the matching error handler in overnight.sh
(overnight.sh line 130) to use the complete CSI escape-sequence pattern,
including optional private-mode markers such as ?. Ensure both log paths remove
sequences like ESC[?25l.
Comment on lines
+199
to
+214
| if $tool_name == "Read" then | ||
| empty | ||
| elif $tool_name == "Glob" then | ||
| ($text | non_empty_lines | length) as $n | | ||
| " < found " + ($n | tostring) + | ||
| (if $n == 1 then " file matching \"" else " files matching \"" end) + | ||
| ($tool.input.pattern // "?") + "\"" | ||
| elif $tool_name == "Grep" then | ||
| ($text | non_empty_lines | length) as $n | | ||
| ($tool.input.output_mode == "files_with_matches") as $is_files | | ||
| " < " + ($n | tostring) + | ||
| (if $is_files then (if $n == 1 then " file" else " files" end) | ||
| else (if $n == 1 then " match" else " matches" end) end) + | ||
| " for \"" + ($tool.input.pattern // "?") + "\"" | ||
| elif $tool_name == "Bash" and is_pure_inspect($tool.input.command // "") then | ||
| empty |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 4 'is_error|tool_name == "Read"|is_pure_inspect' format-stream.jq overnight.shRepository: c0d3ster/nightlight
Length of output: 2951
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- format-stream.jq relevant sections ---'
sed -n '1,80p;110,145p;175,235p' format-stream.jq
echo
echo '--- overnight.sh relevant sections ---'
sed -n '100,135p' overnight.shRepository: c0d3ster/nightlight
Length of output: 9127
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '{
"type":"user","message":{"content":[{"type":"tool_result","tool_use_id":"1","is_error":true,"content":"Read failed: missing file at /tmp/missing.txt"}]}}
' | jq -r -f format-stream.jq
printf '%s\n' '{
"type":"user","message":{"content":[{"type":"tool_result","tool_use_id":"2","is_error":true,"content":"bash -i exited 1"}], "previous_event_id":"old"}}, {
"type":"assistant","message":{"content":[{"type":"tool_use","id":"1","name":"Read","input":{"file":"/tmp/missing.txt"}}]}
' | jq -n --slurp -r -f format-stream.jqRepository: c0d3ster/nightlight
Length of output: 283
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '{
"type":"assistant","message":{"content":[{"type":"tool_use","id":"1","name":"Read","input":{"file":"/tmp/missing.txt"}}]}
},{
"type":"user","message":{"content":[{"type":"tool_result","tool_use_id":"1","is_error":true,"content":"Read failed: missing file at /tmp/missing.txt"}]}}
' | jq -r -f format-stream.jq
printf '%s\n' '{
"type":"assistant","message":{"content":[{"type":"tool_use","id":"2","name":"Bash","input":{"command":"cat /tmp/missing.txt"}}]}
},{
"type":"user","message":{"content":[{"type":"tool_result","tool_use_id":"2","is_error":true,"content":"cat: /tmp/missing.txt: No such file or directory"}]}}
' | jq -r -f format-stream.jq
printf '%s\n' '{
"type":"user","message":{"content":[{"type":"tool_result","tool_use_id":"3","is_error":true,"content":"permission denied"}]}}
' | jq -r -f format-stream.jqRepository: c0d3ster/nightlight
Length of output: 400
Expose failed inspection tool errors before suppression.
When a correlated tool_result has .is_error == true, the Read and pure-inspection Bash branches currently emit no live output while overnight.sh only appends the error after stream processing finishes. Let failed inspection results write through, e.g. in a shared first check, so the live readable log does not hide actionable tool failures.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@format-stream.jq` around lines 199 - 214, Update the tool-result branching in
format-stream.jq to handle .is_error == true before the Read and pure-inspection
Bash suppression branches. Write failed correlated inspection results through to
the live readable output, while preserving suppression for successful Read and
pure-inspection Bash results and existing formatting for Glob and Grep.
c0d3ster
added a commit
that referenced
this pull request
Aug 6, 2026
- #11 [stack: solo] Make DB schema/migration generation NEEDS HUMAN - Amend #2's NEEDS HUMAN trial to require a cache-read cost comparison against the 2026-08-06 baseline before adopting split-per-task as default - nextTaskNumber -> 12 (only #11 was formally assigned here; the out-of-band format-stream.jq log fix, PR #21, was never written into TASKS.md under a number, so it never consumed one) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
c0d3ster
added a commit
that referenced
this pull request
Aug 6, 2026
- #11 [stack: solo] Make DB schema/migration generation NEEDS HUMAN - Amend #2's NEEDS HUMAN trial to require a cache-read cost comparison against the 2026-08-06 baseline before adopting split-per-task as default - nextTaskNumber -> 12 (only #11 was formally assigned here; the out-of-band format-stream.jq log fix, PR #21, was never written into TASKS.md under a number, so it never consumed one) Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
\rbytes from tool output, fixing garbled colored text and phantom blank lines in the readable.logReadresults, pure file-inspection Bash chains (cat/ls/echo/head/tail/find/tasklist/ps), andgit diffbodiesGlob/Grep(and Bash-invokedgrep) show match counts,git status --shortshows a per-status-code count,git commitdrops lefthook/commitlint decoration down to just the commit line + diffstat, vitest/tsc/eslint results extract just the failures/errors/summary, and bracketed-status tool output (codegen, dev-server startup) collapses to its final line<prefix instead of falling back to column 0overnight.shnow appends genuine tool/harness errors (is_errorresults) to the existing stderr log and deletes it if nothing landed thereTest plan
logs/*.jsonl→.logpairs after every change and diffed the output by hand against dozens of real examples (vitest failures, tsc/eslint errors, git add/commit/status/diff, Glob/Grep, dev server startup, chained Bash commands)\rbytes remain in any regenerated logcat file | grep patternstill shows its real output;find src | sortchains aren't swallowed)logs/is gitignored, so this PR touches onlyformat-stream.jqandovernight.sh🤖 Generated with Claude Code
Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com
Summary by CodeRabbit
New Features
Bug Fixes