fix(normalize): strip Claude Code <local-command-*> tags and ANSI escapes

[lucagentile]

What does this PR do?

Fixes #1333.

Extends mempalace/normalize.py:strip_noise() to remove three Claude Code transcript artifacts that currently survive into stored drawers:

1. The remainder of the slash-command tag envelope — <local-command-caveat>, <command-args>, <local-command-stdout> — that Claude Code injects on every slash-command invocation. strip_noise() previously covered only <command-name> and <command-message>; the other three slipped through, leaving drawers with bare <command-args></command-args> remnants and whole <local-command-stdout> payloads.
2. ANSI escape sequences (CSI + OSC) captured verbatim from Bash tool_result content. These bloat embeddings (each escape is several BPE tokens of garbage) and garble semantic search when colored CLI output (e.g. /context, /help, anything piped through a modern tool) lands in a transcript.

Code change (mempalace/normalize.py)

• Adds command-args, local-command-caveat, local-command-stdout to _NOISE_TAGS. They flow through the existing _tag_pattern builder, so they inherit the same line-anchoring + blank-line-bounded body guarantee — a stray unclosed tag in one message still cannot eat content from neighbouring messages.
• Adds two pre-compiled regexes:
  • _ANSI_CSI_RE matches the standard CSI shape ESC [ params interm final covering SGR colors, cursor moves, screen ops.
  • _ANSI_OSC_RE matches ESC ] ... BEL or ESC ] ... ESC \ for terminal title (OSC 0/2) and hyperlinks (OSC 8).
  • Both are anchored on the literal ESC byte (\x1b), so user prose that mentions [1m] or ]8;; by name never matches.
• Applies both ANSI sweeps inside strip_noise() after tag/line/chrome stripping and before the existing blank-line collapse. Tag-wrapped ANSI is already gone via _NOISE_TAG_PATTERNS, so the ANSI sweep only has to cover standalone ANSI in tool output — no double-strip path needed.

Test change (tests/test_normalize.py)

Eleven new test cases, all under the existing TestStripNoiseRemovesSystemChrome and TestStripNoisePreservesUserContent classes to keep the verbatim-safety boundary discipline visible:

• test_strips_each_known_noise_tag extended to iterate the new tags too.
• test_strips_full_claude_code_slash_command_envelope — integration: real five-tag envelope stripped, real prose after it preserved.
• test_strips_empty_command_args_pair — empty-body shape Claude Code emits when a command takes no args.
• test_strips_ansi_color_codes, test_strips_ansi_truecolor_codes, test_strips_ansi_cursor_moves — CSI coverage including 24-bit truecolor sequences from the /context renderer.
• test_strips_ansi_osc_terminal_title, test_strips_ansi_osc_hyperlink — OSC coverage with both terminator forms (BEL, ST).
• test_strips_ansi_inside_noise_tag_with_tag — proves no double-strip is needed; tag-wrapped ANSI exits with its tag.
• test_user_documents_ansi_escape_by_name — user prose mentioning [1m] / [22m] literally (no ESC byte) is preserved verbatim.
• test_user_mentions_local_command_inline — inline <local-command-caveat> mention in user prose stays intact (line-anchoring guarantee).

How to test

python -m pytest tests/test_normalize.py -v
python -m pytest tests/ -v --ignore=tests/benchmarks
ruff check mempalace/normalize.py tests/test_normalize.py
ruff format --check mempalace/normalize.py

Locally: 1499 passed, 1 skipped on the full suite (no new skips introduced); 127/127 in test_normalize.py (was 117); ruff lint clean; mempalace/normalize.py is format-clean. The pre-existing ruff format --check drift in tests/test_normalize.py is left untouched — out of scope for this fix.

Checklist

• Tests pass (python -m pytest tests/ -v --ignore=tests/benchmarks)
• No hardcoded paths
• Linter passes (ruff check .)

[Qodo-Free-For-OSS]

Hi, This PR changes strip_noise() output (new Claude Code tags removed + ANSI escapes stripped) but does not bump NORMALIZE_VERSION, so file_already_mined() will keep treating existing drawers as current and they will not be rebuilt.

Severity: action required | Category: correctness

How to fix: Bump NORMALIZE_VERSION to 3

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

Normalization behavior changed (extra tag stripping + ANSI stripping) but NORMALIZE_VERSION was not bumped. This prevents already-mined drawers from being rebuilt, leaving existing stored transcripts polluted.

Issue Context

file_already_mined() uses normalize_version as a schema gate. Convo mining typically calls file_already_mined() without mtime checking, so version bumps are required to invalidate previously-mined conversation exports.

Fix Focus Areas

• mempalace/palace.py[42-50]
• mempalace/palace.py[395-416]

What to change

• Increment NORMALIZE_VERSION (e.g., 2 → 3).
• Update/extend the inline version history comment to mention this PR’s changes (Claude Code tag envelope + ANSI stripping).

---

Qodo code review - free for open-source.