fix(rewrite): only rewrite safe final pipeline stages

[TaKO8Ki]

#3171

Summary

• Preserve pipeline producers and intermediate stages, rewriting only a verified-safe final grep, rg, or wc stage.
• Keep pattern-file searches, unsupported commands, malformed/grouped pipelines, and |& pipelines raw.
• Tokenize |& atomically while continuing to check both sides during permission analysis.

Behavior

Automatic rewriting previously could rewrite a pipeline producer, changing the bytes consumed by downstream commands.

This PR changes pipeline rewriting so that:

• Producers and intermediate stages remain unchanged.
• Only a verified-safe final stage may be rewritten.
• grep, rg, and wc are initially eligible.
• grep and rg using -f/--file remain raw because those forms are not yet parsed safely.
• Unsupported, malformed, grouped, and |& pipelines remain raw.
• Commands after independent &&, ||, and ; operators can still be rewritten.
• |& is tokenized atomically while permission checks continue to inspect both sides.

For example:

cargo test | grep FAILED
  -> cargo test | rtk grep FAILED

git log | grep feat | wc -l
  -> git log | grep feat | rtk wc -l

cargo test | tail -50
  -> unchanged

cargo test |& grep FAILED
  -> unchanged

Follow-ups

To keep this PR focused and reviewable, I have a plan to address the following in separate PRs.

• Consider a consumable token iterator or cursor if pipeline analysis grows, so already-processed tokens do not need to be skipped repeatedly.
• Support additional final commands after verifying their stdin, exit-code, interactivity, streaming, and argument-specific behavior.
• Consider replacing rtk_cmd string comparisons and centralized command-specific checks with structured rule-level safety policies such as Never, Always, and Conditional.
• Handle redirects and explicit RTK pipeline producers separately.

Test plan

• cargo fmt --all --check
• cargo clippy --all-targets -- -D warnings
• cargo test --all
• cargo test --test pipeline_stdin_test
• Manual testing: rtk <command> output inspected

Important: This PR targets the develop branch.

[TaKO8Ki]

Supporting |& is necessary for this pr. Bash defines both | and |& as pipeline operators, but RTK previously interpreted |& as | followed by background &. This could produce an incorrect pipeline boundary and an unsafe rewrite. This PR recognizes |& atomically so RTK can distinguish it from a normal | pipeline and conservatively leave the complete stderr pipeline raw.

ref: GNU Bash pipeline documentation.

[KuSh]

Shouldn't we handle 2>&1 the same?

[pszymkowiak]

Super PR 🙌 I tested a lot of cases (30-stage pipes, |&, for/if, heredocs, permission deny/ask/allow) and nothing seems broken.

One small optimization left: statements separated by a bare newline only get the last one's final stage rewritten — \n isn't treated as a clause boundary like ;, so earlier lines' grep/wc/rg finals are missed. Not a correctness issue, just coverage:

git log | grep fix | wc -l      # ← not rewritten
gh pr list | grep OPEN | wc -l   # ← only this one

[aeppling]

Hey @TaKO8Ki

Thanks for addressing this important fix.
Here are some point you may want to look at before we merge so we have a solid MR.

search.rs steam stdin

Use streaming for stdin grep before rewriting final stages, currently buffer-to-EOF delivers 0 bytes on a killed pipe (raw grep: 24KB) and tail -f | rtk grep never outputs.

Not out of scope to modify search.rs, it is pre-requisite so all this work together. Open to other ideas

repro:

tail -f <live.logs> | grep ERROR -> 28,672 bytes 
tail -f <live.logs> | rtk grep ERROR -> 0 bytes  

|& operators

NIce catch on this, but instead of discarding all command can't we treat it as a pipe once final stage stream? Could apply same rule as for |

keep rewrite for display tail

We could keep rewriting producers before display-only tails (head/cat/tail without -f), the agent is still the reader there, so it's safe.

cargo test 2>&1 | tail -50     ->  rtk cargo test 2>&1 | tail -50  
git log --oneline | head -20   ->  rtk git log --oneline | head -20

drop wc from the safe finals

rtk wc -l is byte-identical to wc -l (zero savings) and bare rtk wc reshapes the output. Suggest keeping the list to grep/rg and growing it one measured entry at a time.

[aeppling]

lgtm :)

[KuSh]

LGTM!

[KuSh]

nitpick: would be great to have default to avoid having to rewrite every instances each time we add an entry