Skip to content

refactor(tests): annotation syntax with non-ASCII sigils - #537

Merged
16bit-ykiko merged 6 commits into
mainfrom
refactor/annotation-syntax
Jul 21, 2026
Merged

16bit-ykiko merged 6 commits into
mainfrom
refactor/annotation-syntax

Conversation

@16bit-ykiko

Copy link
Copy Markdown
Member

Why

The test annotation grammar reserved $ and @, both of which collide with real content that tests need to contain:

  • @param[in] matches the @key[...] annotation shape and destroys the input; a bare Doxygen tag like @brief aborts assertion-enabled builds. Net effect: no test source could contain Doxygen comments — the hover documentation fixtures avoid them entirely today.
  • $ collides with LSP snippet placeholders (${1:...}), $/cancelRequest, and ${workspace} config variables.
  • The [...] range delimiter collides with C++ brackets: the balance heuristic cannot express a range containing unbalanced brackets (lambda captures, partial subscripts).

New grammar

The grammar now reserves no ASCII character at all — sigil § (U+00A7), range delimiters ⟦/⟧ (U+27E6/E7):

§              nameless point       §(name)          named point
§⟦...⟧         nameless range       §(name)⟦...⟧     named range
  • Ranges nest (stack-based); the bracket-balance heuristic is gone, so unbalanced brackets inside a range are now expressible.
  • §(name) requires an identifier name, keeping a nameless point directly before real parentheses unambiguous (§()).
  • Malformed annotations abort in every build mode (assert-only guards would let NDEBUG wrap past npos and loop forever).

Also introduces test/snap_region.h: whole-line /// <snap:begin [name]> / /// <snap:end [name]> markers plus a containment filter, for snapshot fixtures that only want results from a marked region.

Migration

~750 annotations across 40 files converted mechanically. A semantic differ replayed the old parser against the old files and the new parser against the new files and verified byte-identical stripped sources and annotation tables. Three findings from that process, all fixed:

  • Four sites in selection tests relied on an old-parser quirk where $() meant point-plus-literal-parens; they now use the explicit §()() form.
  • Three commented-out selection cases carried annotations that would have eaten real source parens if ever enabled; re-annotated correctly.
  • One signature-help cursor point had been misclassified as a literal $ during auditing.

Tests

  • New framework coverage (previously zero): 26 cases for the annotation parser and snap-region helpers, with hand-computed byte offsets, UTF-8 passthrough, and a Doxygen/snippet passthrough case.
  • Unit: 1052 passed / 12 pre-existing skips on both RelWithDebInfo and Debug; Integration: 300; Smoke: 3/3.

The old grammar reserved $ and @, which collide with LSP snippet
placeholders (${1:), $/cancelRequest, ${workspace} config vars and
Doxygen tags (@PARAM[in] matches @key[...] and asserts on bare @word).
The new grammar reserves no ASCII character at all:

    §              nameless point       §(name)          named point
    §⟦...⟧         nameless range       §(name)⟦...⟧     named range

Ranges nest via a stack (the bracket-balance heuristic is gone, so
unbalanced brackets inside a range are now expressible), stray sigils
fail loudly, and §(name) requires an identifier so a nameless point
directly before real parentheses stays unambiguous (§()).

Also adds test/snap_region.h: whole-line /// <snap:begin>/<snap:end>
markers plus a containment filter, for snapshot fixtures that only want
a marked region's results.

All ~750 annotations across 40 files migrated mechanically; a semantic
differ verified byte-identical stripped sources and annotation tables
against the old parser (four sites in selection_tests relied on the old
$() quirk and were fixed, one signature-help point was recovered).
Review found the assert-only guards let NDEBUG builds wrap i past npos
on an unterminated §( (infinite loop) and pop an empty stack on a stray
⟧ (UB). Malformed annotations now LOG_FATAL unconditionally. Also adds
the review-suggested coverage: digit-only names, § at EOF, empty range
body, end marker without trailing newline, empty snap region.
@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c96e6e91-c785-437b-af8b-d587eab489ab

📥 Commits

Reviewing files that changed from the base of the PR and between b46a2f7 and 73242cb.

📒 Files selected for processing (1)
  • tests/unit/test/snap_region.h
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/unit/test/snap_region.h

📝 Walkthrough

Walkthrough

The annotation framework now uses § points and ⟦…⟧ ranges. The parser supports nested ranges and validation, snapshot-region utilities were added, and affected hover, feature, semantic, index, and server tests were migrated.

Changes

Annotation grammar and test infrastructure

Layer / File(s) Summary
Annotation parser and snapshot regions
tests/unit/test/annotation.cpp, tests/unit/test/snap_region.h, tests/unit/test/annotation_tests.cpp
Implements the new annotation grammar, snapshot-region extraction/filtering, structural validation, and corresponding tests.
Hover fixtures
tests/data/hover/*
Replaces legacy hover markers throughout the fixture suite with the new § syntax.
Feature and semantic fixtures
tests/unit/compile/*, tests/unit/feature/*, tests/unit/semantic/*
Migrates directive, completion, document-link, folding, hover, inlay-hint, semantic-token, signature-help, and selection inputs.
Index and server fixtures
tests/unit/index/*, tests/unit/server/*
Migrates symbol, preamble, USR, freshness, and overlay test annotations to the new point/range format.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • clice-io/clice#314: Adds the prior annotation parser implementation that this migration replaces.
  • clice-io/clice#377: Updates the related annotation infrastructure used by the new grammar.
  • clice-io/clice#452: Migrates related hover test infrastructure to the § marker syntax.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.35% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: refactoring test annotation syntax to non-ASCII sigils.
Description check ✅ Passed The description matches the changeset and explains the new annotation grammar, migration, and added snapshot-region support.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/annotation-syntax

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7863e7b1ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/unit/test/snap_region.h Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/unit/feature/document_link_tests.cpp (1)

47-52: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove stray nameless point markers.

The trailing § markers placed just before the range closing markers (⟧) define unused nameless points. Since nameless_points() is not utilized in these test cases, they are functionally harmless but likely unintentional leftovers from the marker syntax migration.

  • tests/unit/feature/document_link_tests.cpp#L47-L52: remove the trailing § from the include arguments (e.g., ⟦"test.h"§⟧ -> ⟦"test.h"⟧).
  • tests/unit/feature/document_link_tests.cpp#L89-L89: remove the trailing § from ⟦HEADER§⟧.
  • tests/unit/feature/document_link_tests.cpp#L103-L103: remove the trailing § from ⟦"bytes.bin"§⟧.
  • tests/unit/feature/document_link_tests.cpp#L118-L118: remove the trailing § from ⟦"data.bin"§⟧.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/feature/document_link_tests.cpp` around lines 47 - 52, Remove the
unused trailing nameless-point markers from the marker ranges in
tests/unit/feature/document_link_tests.cpp: lines 47-52 include arguments, line
89 HEADER, line 103 "bytes.bin", and line 118 "data.bin"; preserve the
surrounding marker syntax and test content.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/unit/test/annotation.cpp`:
- Around line 81-83: Update the point-annotation insertion at
tests/unit/test/annotation.cpp lines 81-83 to assert that insertion succeeds
instead of silently ignoring duplicate keys; update the range-annotation
insertion at lines 91-94 to assert success and explicitly reject a second
nameless range keyed by the empty string, preserving the fail-loudly behavior
for duplicate annotation keys.

---

Nitpick comments:
In `@tests/unit/feature/document_link_tests.cpp`:
- Around line 47-52: Remove the unused trailing nameless-point markers from the
marker ranges in tests/unit/feature/document_link_tests.cpp: lines 47-52 include
arguments, line 89 HEADER, line 103 "bytes.bin", and line 118 "data.bin";
preserve the surrounding marker syntax and test content.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 444f980c-8a0b-413b-ad77-45613a8b3e98

📥 Commits

Reviewing files that changed from the base of the PR and between 5526782 and f93593c.

📒 Files selected for processing (44)
  • tests/data/hover/.clang-format
  • tests/data/hover/attributes.cpp
  • tests/data/hover/auto.cpp
  • tests/data/hover/basics.cpp
  • tests/data/hover/callee_args.cpp
  • tests/data/hover/concepts.cpp
  • tests/data/hover/decltype.cpp
  • tests/data/hover/docs.cpp
  • tests/data/hover/expressions.cpp
  • tests/data/hover/fields.cpp
  • tests/data/hover/functions.cpp
  • tests/data/hover/getter_setter.cpp
  • tests/data/hover/lambdas.cpp
  • tests/data/hover/misc.cpp
  • tests/data/hover/no_hover.cpp
  • tests/data/hover/no_hover_errors.cpp
  • tests/data/hover/pass_types.cpp
  • tests/data/hover/spaceship.cpp
  • tests/data/hover/tag_decls.cpp
  • tests/data/hover/template_params.cpp
  • tests/data/hover/templates.cpp
  • tests/data/hover/this_expr.cpp
  • tests/data/hover/using_decls.cpp
  • tests/data/hover/values.cpp
  • tests/data/hover/variables.cpp
  • tests/unit/compile/directive_tests.cpp
  • tests/unit/feature/code_completion_tests.cpp
  • tests/unit/feature/document_link_tests.cpp
  • tests/unit/feature/folding_range_tests.cpp
  • tests/unit/feature/hover_tests.cpp
  • tests/unit/feature/inlay_hint_tests.cpp
  • tests/unit/feature/semantic_tokens_tests.cpp
  • tests/unit/feature/signature_help_tests.cpp
  • tests/unit/index/index_query_tests.cpp
  • tests/unit/index/merged_index_tests.cpp
  • tests/unit/index/preamble_state_tests.cpp
  • tests/unit/index/tu_index_tests.cpp
  • tests/unit/index/usr_tests.cpp
  • tests/unit/semantic/selection_tests.cpp
  • tests/unit/server/query_freshness_tests.cpp
  • tests/unit/server/query_overlay_tests.cpp
  • tests/unit/test/annotation.cpp
  • tests/unit/test/annotation_tests.cpp
  • tests/unit/test/snap_region.h

Comment thread tests/unit/test/annotation.cpp Outdated
try_emplace silently kept the first binding, so a duplicate §(name) or
a second nameless range dropped an annotation without a trace.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b46a2f7112

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/unit/test/snap_region.h Outdated
@16bit-ykiko
16bit-ykiko merged commit 26c4643 into main Jul 21, 2026
30 checks passed
@16bit-ykiko
16bit-ykiko deleted the refactor/annotation-syntax branch July 21, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant