[thehoff]

What

Closes #157. Adds contextcrawler read <file> --intent "<terms>" for surgical lexical extraction on large files.

When --intent is provided AND file > 5 KB AND the splitter finds ≥ 3 sections AND at least one section scores above zero, the output becomes:

<metadata banner>
<head bookend>          (omitted on files < 60 lines, byte-capped at 1 KB)
<top-scoring sections in original document order>
<omission count>
<tail bookend>          (same gating as head)

Otherwise falls through to the existing render path — no regression on small files, no regression when intent omitted.

Why

read ranks #4 by absolute leaked tokens per the v0.1.10 audit (1838 runs, 2.5M leaked, 87.3% saved). The current symmetric head/tail cap works for medium files but degrades on large files where the relevant content sits in the middle.

How

Section splitters covered in v1:

• Markdown (.md, .markdown) — heading lines, skips fenced code blocks
• Rust (Language::Rust) — top-level item starts (fn, impl, struct, enum, trait, mod, type, const, static), with vis/async/unsafe qualifiers handled
• TOML / Cargo.lock (.toml, .lock) — lines starting with [
• Unknown — paragraph split on blank-line runs

Scoring is lexical: per-term occurrence count in body plus heading hits weighted 5×. Tokenisation is case-insensitive, alphanumeric + underscore only. Stopwords dropped. No stemming, no LLM — preserves the zero-async, fast-startup posture.

Live result

contextcrawler read Cargo.lock                            → 0% saved (baseline)
contextcrawler read Cargo.lock --intent "rusqlite chrono" → 96.1% saved

From 2316 lines to 114 lines, with the matching rusqlite and chrono dependency sections preserved verbatim.

Tests

16 new (3 commits, 2691/2691 pass total):

• tokenize_drops_stopwords_and_lowercases
• tokenize_handles_punctuation
• tokenize_only_stopwords_yields_empty
• markdown_splitter_finds_sections
• markdown_splitter_ignores_headings_in_fenced_code
• rust_splitter_handles_visibility_and_qualifiers
• rust_splitter_does_not_split_inner_fns
• toml_splitter_array_of_tables
• extract_returns_none_for_small_input
• extract_returns_none_when_no_section_matches
• extract_keeps_matching_section_and_drops_others
• extract_preserves_head_and_tail_bookends
• extract_achieves_meaningful_reduction
• extract_on_real_cargo_lock_meets_savings_target — fixture-backed Cargo.lock benchmark, asserts ≥ 80% savings on intent="rusqlite chrono" plus content sanity (matched terms must appear in output)
• take_capped_does_not_panic_on_multibyte_overflow — UTF-8 byte safety, 100 emoji at cap=50
• take_capped_emits_nothing_when_cap_is_tiny — cap < marker.len edge case

Commits

SHA	Title
dd3eb27	feat(read): --intent for surgical lexical extraction on large files (#157)
4eae758	harden(read): UTF-8 byte cap + Cargo.lock fixture-backed savings (#157)
6a4b5f3	brand(logo): swap to exec()-flavoured logo + update README alt text

Risk

• Opt-in flag: zero behaviour change when --intent is omitted.
• Fall-through fallback: extract_for_intent returns None for too-small / no-match / unsupported-format inputs, caller drops to existing render path.
• No async, no LLM, no network: stdlib string ops only.
• Logo swap: drop-in 1254×1254 replacement, README alt-text updated to match.