Skip to content

witness gate: requiring the :: payload silently drops de-marked markers that have no payload, and the changelog claims otherwise - #353

Closed
jaylfc wants to merge 1 commit into
masterfrom
exec/tsk-y54vhk
Closed

witness gate: requiring the :: payload silently drops de-marked markers that have no payload, and the changelog claims otherwise#353
jaylfc wants to merge 1 commit into
masterfrom
exec/tsk-y54vhk

Conversation

@jaylfc

@jaylfc jaylfc commented Aug 18, 2026

Copy link
Copy Markdown
Owner

CARD TITLE (intent, not commit subject): witness gate: requiring the :: payload silently drops de-marked markers that have no payload, and the changelog claims otherwise

Autonomous build of board card tsk-y54vhk.

  • Change _NEAR_MISS_RE from (?=.::) to (?=[^:]:) so de-marked markers
    whose separator is broken but that still contain a colon after the
    separator are caught, fixing a silent regression where
    '# WITNESS: ' with no :: payload exited 0.
  • Keep ordinary prose without a colon invisible to the near-miss detector.
  • Note in the docstring that prose quoting the marker syntax (path::token)
    still trips the near-miss detector as a known residual.
  • Add Arm D (ZWSP, no :: payload) to test_near_miss_regex_spares_prose_arms
    so all four required arms live in one test.
  • Update _DEMARKED_MARKER_EXEMPTION line numbers to match the docstring.
  • Add changelog fragment tsk-y54vhk-witness-gate-near-miss-fix.md.

Files:
.../tsk-2k55kq-witness-gate-near-miss-hardening.md | 8 +++
.../tsk-y54vhk-witness-gate-near-miss-fix.md | 12 ++++
scripts/check_witness_token.py | 28 +++++---
tests/test_witness_gate.py | 78 ++++++++++++++++++++++
4 files changed, 118 insertions(+), 8 deletions(-)

…t changelog

- Change _NEAR_MISS_RE from (?=.*::) to (?=[^:]*:) so de-marked markers
  whose separator is broken but that still contain a colon after the
  separator are caught, fixing a silent regression where
  '# WITNESS<ZWSP>: <path>' with no :: payload exited 0.
- Keep ordinary prose without a colon invisible to the near-miss detector.
- Note in the docstring that prose quoting the marker syntax (path::token)
  still trips the near-miss detector as a known residual.
- Add Arm D (ZWSP, no :: payload) to test_near_miss_regex_spares_prose_arms
  so all four required arms live in one test.
- Update _DEMARKED_MARKER_EXEMPTION line numbers to match the docstring.
- Add changelog fragment tsk-y54vhk-witness-gate-near-miss-fix.md.
@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jaylfc, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Limit details: You’ve used all 1 included review currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 43e27210-99e2-45bb-b7f3-11de80dacc5a

📥 Commits

Reviewing files that changed from the base of the PR and between 72ade2c and 4ccad63.

📒 Files selected for processing (4)
  • changelog.d/tsk-2k55kq-witness-gate-near-miss-hardening.md
  • changelog.d/tsk-y54vhk-witness-gate-near-miss-fix.md
  • scripts/check_witness_token.py
  • tests/test_witness_gate.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gitar-bot

gitar-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

Important

You are using the Gitar free plan. Upgrade to unlock code review, CI analysis, auto-apply, custom automations, and more.

Gitar

# They are intentionally de-marked and must not be reported; every other line
# in the same file (e.g. an appended genuine marker) still is. Greppable name.
_DEMARKED_MARKER_EXEMPTION = {
"scripts/check_witness_token.py": frozenset({4, 21, 38}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Hardcoded line numbers in _DEMARKED_MARKER_EXEMPTION

The frozenset {4, 21, 38} is hardcoded to match specific docstring lines. If the docstring in this file is modified in the future, these line numbers will drift, causing the near-miss detector to either falsely flag docstring examples or fail to flag genuine near-misses. Consider using a more robust mechanism (e.g., detecting de-marked markers dynamically, or adding a marker comment in the docstring).


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

docstring examples are not.
- Corrected the near-miss detector to catch de-marked markers whose payload
is missing, fixing a silent regression introduced when the detector was
tightened to require the ``::`` payload.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Changelog claims old regex required :: payload

The changelog states the old regex required the :: payload, but the diff shows the previous regex was r"#\\s*WITNESS[^:]", which does not require ::. Consider revising the changelog to accurately describe the prior regex behavior.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
scripts/check_witness_token.py 80 Hardcoded line numbers in _DEMARKED_MARKER_EXEMPTION will drift if the docstring changes

SUGGESTION

File Line Issue
changelog.d/tsk-y54vhk-witness-gate-near-miss-fix.md 12 Changelog claims old regex required :: payload, but diff shows r"#\\s*WITNESS[^:]"
Files Reviewed (4 files)
  • scripts/check_witness_token.py - 1 issue
  • tests/test_witness_gate.py
  • changelog.d/tsk-2k55kq-witness-gate-near-miss-hardening.md
  • changelog.d/tsk-y54vhk-witness-gate-near-miss-fix.md - 1 issue

Fix these issues in Kilo Cloud


Reviewed by step-3.7-flash · Input: 208.4K · Output: 72.3K · Cached: 1M

@jaylfc

jaylfc commented Aug 18, 2026

Copy link
Copy Markdown
Owner Author

BLOCK — and to be unambiguous: the work in this PR is correct. The regex fix does close the defect it was filed for, verified below with both controls. This is blocked purely because the branch predates #344 and no longer merges. Revision card: tsk-nss5ya.

Measured on a trial merge of 4ccad630 into origin/master at review time. Merge-base 5796bc26.

The fix is right

case                                          BASE   MASTER  PR#353
POSITIVE CONTROL: de-marked, NO :: payload    True   False   True    <- THE DEFECT, fixed
de-marked, WITH :: payload                    True   True    True
NEGATIVE CONTROL: genuine marker              False  False   False   <- must stay False
prose, engages prefix, no colon at all        True   False   False
prose quoting path::token syntax              True   True    True
prose with a plain colon, no ::               True   False   True

Row 1 is the whole point: a de-marked marker carrying no :: payload is silently dropped by master's (?=.*::) and is caught by this PR's (?=[^:]*:). Row 3 confirms a genuine marker is still not flagged as a near-miss.

Probe discipline note: my first pass at this table used prose strings like # a WITNESS marker …, which never engage the #\s*WITNESS prefix at all — they returned False everywhere while measuring nothing. The table above asserts prefix engagement on every case before recording a result.

The blocker — the branch predates #344

CONFLICT (content): Merge conflict in scripts/check_witness_token.py
CONFLICT (content): Merge conflict in tests/test_witness_gate.py

Cut from 5796bc26, which is before #344 merged as 4175efdd. #344 rewrote the same regex and the same exemption structure, so this branch re-implements it and both sides collide:

base 5796bc26  r"#\s*WITNESS[^:]"                 frozenset({"scripts/check_witness_token.py"})
master         r"#\s*WITNESS[^:](?=.*::)"         {"scripts/check_witness_token.py": frozenset({4, 19, 36})}
PR #353        r"#\s*WITNESS[^:](?=[^:]*:)"       {"scripts/check_witness_token.py": frozenset({4, 21, 38})}

The PR's regex is the correct end state; it simply cannot be applied as-is.

Not a problem, checked so it is not mistaken for one: the PR also re-adds changelog.d/tsk-2k55kq-witness-gate-near-miss-hardening.md, already on master. Both copies are byte-identical (md5 c3a8b1f2), so git merges it silently.

The rebase hazard worth naming

The exemption set pins line numbers, and both sets are correct for their own file — they differ by 2 only because this PR adds two comment lines above them:

master {4, 19, 36}                 PR #353 {4, 21, 38}
   4: 'Verifies that every explicit ``# WITNESS<ZWSP>: <test>::<token>`` marker…'
19/21: '    # WITNESS<ZWSP>: tests/test_foo.py::some_grepable_token'
36/38: '(e.g. ``f"# WITNESS<ZWSP>: {TEST}::{TOK}"``) whose interpolated names never'

After the rebase the merged docstring is a third arrangement, so these must be re-derived from the resolved file rather than taken from either side. They fail loud when wrong, so this is safe rather than silent — but it must actually be redone.

A widening I flagged, then measured down to size

Row 6 shows this PR flags a line master does not: prose with a plain colon and no ::, e.g. # WITNESS handling: see docs. The code comment documents only the path::token residual (row 5), which master already shares and so is not new.

Rather than leave that as a worry, I swept the real tree:

scanned 339 tracked files (.py .md .txt .toml .yml .yaml .sh)
lines flagged by BOTH regexes:                              6
flagged by PR#353 but NOT master (new surface):             0
flagged by master but NOT PR#353 (coverage lost):           0

Zero new flags and zero lost coverage on the current tree. So this is not a defect and not a reason to touch the regex — it warrants one accurate sentence in the comment, nothing more. Recording it because the comment currently reads as if path::token were the only residual.

Stated limitations

  • Gates and the full suite were not run to completion: the tree does not merge. They are required on the revised branch (baseline 1617 passed, 12 skipped).
  • I did not hand-resolve the conflicts. They are mechanical, but the resolution is the lane's call and mine would not be what lands.
  • The 339-file sweep covers tracked files with the listed extensions; a new file type carrying such a comment would not have been counted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant