[None][infra] Tighten CBTS testdef anchors for deletion-only test diffs

jenkins/scripts/cbts/rules/README.md

@@ -50,6 +50,11 @@ For each file under `tests/` in the diff:
    - file-level anchor `path` (any line lands at module scope, AST parse
      fails, or the file is unreadable — the latter covers .yaml/.txt/
      .json/etc. data files).
+   Decorator lines count as part of what they decorate: `_scope_start_line`
+   extends each node's range up over its `decorator_list`, so editing
+   `@pytest.mark.parametrize` stays at method level and a class-level
+   `@skip_pre_blackwell` stays at class level instead of dropping to
+   module scope.
 3. `lookup_paths_into_block_filters` calls `find_match_for_path`
    (bidirectional pytest-tree lineage) for each anchor; matches feed
    `block_filters`. For non-`test_*.py` paths, the lookup walks up
@@ -58,13 +63,51 @@ For each file under `tests/` in the diff:
    `unittest/api_stability/references/llmapi.yaml` lifts to
    `unittest/api_stability/`.
 
-`accuracy/references/*.yaml` gets a finer-grained refinement: each
-top-level YAML key is a HF model name, mapped (via AST scan of
-`accuracy/test_*.py` for `MODEL_NAME = "<hf>"` literals) to test
-classes. A diff under `meta-llama/Llama-3.1-8B-Instruct:` narrows to
-`accuracy/test_*.py::TestLlama3_1_8BInstruct` rather than the whole
-`accuracy/` subtree. Models with no matching test class fall back to
-the dir-level anchor.
+### Deletion-only diffs
+
+Deleted lines have no post-image position. `iter_diff_post_line_numbers`
+anchors them to the next surviving line, which can land outside the scope
+they were deleted from — between two classes for a `.py` file, or on the
+next model section for a reference YAML. Rather than widening to
+file/dir level, both paths re-read the deleted side from the diff's own
+pre-image view (`iter_diff_pre_image`: context + `-` lines, in-hunk
+order), where a deleted class's `class` statement or a deleted section's
+key line is directly visible:
+
+- `.py`: `_recover_deleted_scope_anchors` runs only after post-image
+  mapping already failed, so modification diffs keep their finer
+  method-level anchors. `_py_class_scopes_from_deletions` attributes at
+  class level — a method-level walk would have to guess which `def` a
+  stray decorator line belongs to.
+- reference YAML: `_yaml_top_keys_from_deletions` reads `-` lines,
+  `iter_diff_added_post_line_numbers` reads `+` lines off the post-image.
+
+Both return `None` — forcing the file-level fallback — when a `-` line's
+owning scope is not visible inside its hunk, since only 3 context lines
+are guaranteed. Pre-image *line numbers* are never used:
+`strip_noop_diff_lines` drops blank / comment-only `-` lines, which
+shifts every later pre-image number.
+
+### accuracy/references/*.yaml
+
+Top-level keys name what a section is a reference for, and resolve to
+anchors under `accuracy/test_*.py` so the lineage walk matches every
+parametrization:
+
+- HF model name (15 of the 16 files) → test classes carrying that
+  `MODEL_NAME = "<hf>"` literal. A diff under
+  `meta-llama/Llama-3.1-8B-Instruct:` narrows to
+  `accuracy/test_*.py::TestLlama3_1_8BInstruct` rather than the whole
+  `accuracy/` subtree.
+- `TestC::test_m` id (`acceptance_length.yaml` only) → that method in
+  every module defining `TestC`.
+
+Resolving to *no* anchor is a zero-impact claim, not a failure, when the
+changed keys are also absent from the post-PR YAML: that is the shape of
+a test-pruning PR, which drops a reference section and its test in one
+commit, so nothing left in the tree can read it. Keys that survive in
+the YAML but resolve to no test keep the dir-level fallback — some test
+this rule failed to resolve may still read them.
 
 Outcomes:
 
@@ -75,10 +118,11 @@ Outcomes:
   - basename is conftest / `__init__` / a helper / data file: unhandled
     — Selector reports it and falls back. Could be implicitly imported
     (top-level conftest, sys-path helpers, test input fixtures).
-- Path is in-namespace but no YAML-covered ancestor exists at any
-  walk-up level: claimed as no-narrow contribution (`scope=noop` if
-  all paths are like this; a miss-note in the reason on partial-narrow
-  runs).
+- Path is in-namespace but its anchors match no YAML entry — no
+  YAML-covered ancestor at any walk-up level, or a zero-impact claim
+  from the accuracy-reference path: claimed as no-narrow contribution
+  (`scope=noop` if all paths are like this; a miss-note in the reason
+  on partial-narrow runs).
 - Block-filter coverage ≥ `BLAST_RADIUS_FRACTION` (0.8) of total YAML
   blocks: `scope=None` (rule cannot usefully narrow — fallback).
 - `sanity_relevant` / `perfsanity_relevant` follow from the matched
@@ -278,7 +322,9 @@ in `out_of_scope_rule.py` list the patterns.
 | Helper | Used by | Purpose |
 |---|---|---|
 | `iter_diff_changes(diff)` | waives, testlist | Yields `(sign, body)` for each `+`/`-` content line. |
-| `iter_diff_post_line_numbers(diff)` | testdef | Yields post-image (`+`) line numbers for AST scope mapping. |
+| `iter_diff_post_line_numbers(diff)` | testdef | Yields post-image line numbers for AST scope mapping; `-` lines anchor to the next surviving line. |
+| `iter_diff_added_post_line_numbers(diff)` | testdef | Same, but `+` lines only — for callers reading meaning off the post-image. |
+| `iter_diff_pre_image(diff)` | testdef | Yields `(sign, body)` reconstructing each hunk's pre-image (`-` + context); `("@", "")` marks a hunk boundary. |
 | `lookup_ids_into_block_filters` | waives, testlist | Runs `find_match_for_waive` over a set of test ids; returns block_filters and miss set. |
 | `lookup_paths_into_block_filters` | testdef | Runs `find_match_for_path` over a set of anchors; returns block_filters and miss set. |
 | `resolve_affected_stages` | all narrowing rules | Maps `block_filters` keys to stage names via `stages_by_yaml_stem`. |

jenkins/scripts/cbts/rules/_helpers.py

@@ -116,6 +116,59 @@ def iter_diff_post_line_numbers(diff: str) -> set[int]:
     return out
 
 
+def iter_diff_added_post_line_numbers(diff: str) -> set[int]:
+    """Post-PR line numbers (1-indexed) touched by `+` lines only.
+
+    Unlike `iter_diff_post_line_numbers`, `-` lines are excluded rather
+    than anchored to the next surviving line. Callers that read meaning
+    from the post-image (YAML section, AST scope) need this: a deleted
+    line's anchor can land in a different section than the one it was
+    deleted from.
+    """
+    out: set[int] = set()
+    new_line = 0
+    for line in diff.splitlines():
+        m = _HUNK_HEADER_RE.match(line)
+        if m is not None:
+            new_line = int(m.group(1))
+            continue
+        if not line or line.startswith(("+++", "---")):
+            continue
+        sign = line[0]
+        if sign == "+":
+            out.add(new_line)
+            new_line += 1
+        elif sign != "-":
+            new_line += 1
+    return out
+
+
+def iter_diff_pre_image(diff: str) -> Iterator[tuple[str, str]]:
+    """Yield `(sign, body)` for the pre-image content of each hunk.
+
+    Context and `-` lines together reconstruct what the file looked like
+    before the change, in order, within each hunk. `sign` is `-` for
+    deleted lines and ` ` for context. A `@@` hunk header yields
+    `("@", "")` so callers can reset any positional state they track —
+    pre-image continuity does not hold across hunk boundaries.
+
+    Blank/comment-only additions may appear as context despite being absent before the change.
+    Line numbers are deliberately not reported: `strip_noop_diff_lines`
+    drops blank / comment-only `-` lines, which shifts every later
+    pre-image line number. Callers must derive meaning from the
+    in-hunk ordering instead.
+    """
+    for line in diff.splitlines():
+        if line.startswith("@@"):
+            yield "@", ""
+            continue
+        if not line or line.startswith(("+++", "---")):
+            continue
+        sign = line[0]
+        if sign in ("-", " "):
+            yield sign, line[1:]
+
+
 def lookup_ids_into_block_filters(
     yaml_index: YAMLIndex,
     test_ids: Iterable[str],