Skip to content

feat(tests): add compile_with_modules helper to Tester - #420

Merged
16bit-ykiko merged 2 commits into
mainfrom
feat/tester-compile-with-modules
Apr 11, 2026
Merged

16bit-ykiko merged 2 commits into
mainfrom
feat/tester-compile-with-modules

Conversation

@16bit-ykiko

@16bit-ykiko 16bit-ykiko commented Apr 9, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Add add_module() and compile_with_modules() to the Tester test framework
  • Supports both separate add_module() calls and single-string #[filename] syntax via add_files()
  • Automatically scans module dependencies with scan_precise, topologically sorts, builds PCMs in order, then compiles the main file
  • Temporary PCM files cleaned up automatically in destructor
  • Migrated ModuleImport and ModuleReexport semantic tokens tests to use the new API

Test plan

  • All 505 unit tests pass
  • All 113 integration tests pass
  • All 2 smoke tests pass

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Centralized, module-aware test compilation with automatic module discovery, dependency ordering, and cycle detection.
    • Unified "compile with modules" flow; tests now add module sources directly and no longer manage temporary module artifacts manually.
    • Reduced duplicated compile/diagnostic logic and improved cleanup of generated artifacts.

@coderabbitai

coderabbitai Bot commented Apr 9, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds module-aware compilation to the test harness: Tester now records module sources, scans and topologically orders module dependencies, emits per-module PCM files to temporary paths, then compiles content using an overlay VFS and the generated PCMs. Tests were updated to use this helper instead of managing .pcm files manually.

Changes

Cohort / File(s) Summary
Test Infrastructure - Header
tests/unit/test/tester.h
Add ModuleFile type, module_files and pcm_paths members, ~Tester() declaration, add_module(), compile_with_modules(), and try_compile() declarations.
Test Infrastructure - Impl
tests/unit/test/tester.cpp
Implement ~Tester() to remove PCM temp files; add base_cc1_args(); centralize compile flow into try_compile(); implement compile_with_modules() to scan modules, topo-sort dependencies with cycle detection, emit per-module PCMs to temp files (tracking pcm_paths), rebuild overlay VFS and PCM map, then invoke compilation; update prepare() and clear() to manage module state.
Tests - Feature
tests/unit/feature/semantic_tokens_tests.cpp
Remove manual PCM creation and explicit -fmodule-file=... forwarding; add module and main sources into the tester and rely on compile_with_modules() for module build and provisioning; remove temp-file cleanup.

Sequence Diagram

sequenceDiagram
    participant Test as Test Case
    participant Tester
    participant Scanner as Module Scanner
    participant Compiler as Clang -cc1
    participant OVFS as Overlay VFS
    participant FS as Temp PCM Storage

    Test->>Tester: add module sources + main
    Test->>Tester: compile_with_modules()

    Tester->>Scanner: scan sources for module names & deps
    Scanner-->>Tester: module names & dependency graph

    Tester->>Tester: topo-sort modules (DFS, detect cycles)

    loop for each module (topo order)
        Tester->>Compiler: compile ModuleInterface
        Compiler-->>FS: emit PCM (temp path)
        Tester->>OVFS: register PCM path & mapping
    end

    Tester->>Compiler: compile main using OVFS + PCMs
    Compiler-->>Tester: compilation result
    Tester-->>Test: return success/failure
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰 I scanned each module, nibbling names so neat,
I stored PCMs in temp-holes, tidy and petite,
Topo hops and cycle checks, a careful little chase,
Overlay paths guide compilers to the right place,
Tests sprout like carrots — crunchy, swift, and sweet.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(tests): add compile_with_modules helper to Tester' is concise, specific, and clearly summarizes the main change: adding a new helper function to the test framework.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/tester-compile-with-modules

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/unit/test/tester.cpp`:
- Around line 219-243: A temporary PCM created via createTemporaryFile
(pcm_path) is only recorded into pcm_paths after successful compile, so failures
leak files; update the code around the compile sequence to delete/unlink the
temporary file whenever pcm_path.has_value() and compilation fails or an early
return occurs (log errors if unlink fails), and ensure built_pcms entries are
not left pointing at removed files; additionally, modify clear() to iterate
pcm_paths, delete/unlink each file before calling pcm_paths.clear() (and remove
corresponding entries in built_pcms) so all temporary PCM files are cleaned up
on both failure paths and when clearing state; reference symbols: pcm_path,
pcm_paths, built_pcms, createTemporaryFile, compile(builder.params), and
clear().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3f9ddb24-f250-40a2-ad72-7b5219cb6f4d

📥 Commits

Reviewing files that changed from the base of the PR and between 8bafaa8 and 3ed796a.

📒 Files selected for processing (3)
  • tests/unit/feature/semantic_tokens_tests.cpp
  • tests/unit/test/tester.cpp
  • tests/unit/test/tester.h

Comment thread tests/unit/test/tester.cpp
@16bit-ykiko
16bit-ykiko force-pushed the feat/tester-compile-with-modules branch 2 times, most recently from 06e118f to 9077c31 Compare April 9, 2026 16:10

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
tests/unit/test/tester.cpp (1)

202-206: ⚠️ Potential issue | 🟠 Major

Temporary PCM files still leak on failure and clear().

At Line 202, a temp PCM can be created and then returned early on failure (Line 217) before it is tracked; and Line 373 clears pcm_paths without deleting files. This leaves orphaned temp files.

💡 Proposed fix
@@
         auto pcm_path = fs::createTemporaryFile("clice", "pcm");
         if(!pcm_path) {
             LOG_ERROR("{}", pcm_path.error().message());
             return false;
         }
+        pcm_paths.push_back(*pcm_path);
@@
         auto built = clice::compile(builder.params);
         if(!built.completed()) {
             for(auto& diag: built.diagnostics()) {
                 LOG_ERROR("{}", diag.message);
             }
             return false;
         }
 
         built_pcms.try_emplace(mod.module_name, *pcm_path);
-        pcm_paths.push_back(*pcm_path);
@@
 void Tester::clear() {
@@
-    module_files.clear();
-    pcm_paths.clear();
+    for(const auto& path: pcm_paths) {
+        fs::remove(path);
+    }
+    pcm_paths.clear();
+    module_files.clear();
 }

Also applies to: 217-226, 364-373

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/unit/test/tester.cpp` around lines 202 - 206, The temporary PCM file
created by fs::createTemporaryFile (pcm_path) can leak because you return early
after logging (LOG_ERROR) before adding it to pcm_paths, and pcm_paths.clear()
is called later without deleting files; modify the logic in the test functions
that use pcm_path and pcm_paths so that: 1) immediately register the created
file path with the tracker (pcm_paths) as soon as createTemporaryFile returns
successfully or ensure you delete the file before any early return (references:
pcm_path, LOG_ERROR, createTemporaryFile); 2) replace pcm_paths.clear() with a
proper cleanup routine that iterates pcm_paths and deletes each file (or use an
RAII helper to auto-remove files) so clearing does not leave orphaned temp files
(references: pcm_paths, clear()).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@tests/unit/test/tester.cpp`:
- Around line 202-206: The temporary PCM file created by fs::createTemporaryFile
(pcm_path) can leak because you return early after logging (LOG_ERROR) before
adding it to pcm_paths, and pcm_paths.clear() is called later without deleting
files; modify the logic in the test functions that use pcm_path and pcm_paths so
that: 1) immediately register the created file path with the tracker (pcm_paths)
as soon as createTemporaryFile returns successfully or ensure you delete the
file before any early return (references: pcm_path, LOG_ERROR,
createTemporaryFile); 2) replace pcm_paths.clear() with a proper cleanup routine
that iterates pcm_paths and deletes each file (or use an RAII helper to
auto-remove files) so clearing does not leave orphaned temp files (references:
pcm_paths, clear()).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 26c44833-b838-4c80-b943-e879f1c3ff7c

📥 Commits

Reviewing files that changed from the base of the PR and between 06e118f and 9077c31.

📒 Files selected for processing (3)
  • tests/unit/feature/semantic_tokens_tests.cpp
  • tests/unit/test/tester.cpp
  • tests/unit/test/tester.h
✅ Files skipped from review due to trivial changes (1)
  • tests/unit/feature/semantic_tokens_tests.cpp

Add add_module() and compile_with_modules() to the Tester framework,
enabling concise module tests via either separate add_module() calls
or single-string #[filename] syntax with add_files(). The helper
automatically scans for module dependencies, topologically sorts them,
builds PCMs, and compiles the main file. Temporary PCM files are
cleaned up in the destructor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@16bit-ykiko
16bit-ykiko force-pushed the feat/tester-compile-with-modules branch from 9077c31 to 62aa5dd Compare April 9, 2026 23:04

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/unit/test/tester.cpp`:
- Around line 161-164: The loop that populates name_to_index from modules
currently overwrites previous entries when modules[i].module_name is duplicated;
change the insertion to detect duplicates (e.g., use name_to_index.find(...) or
try_emplace/insert and check the returned pair) and if a duplicate module_name
is found, fail the test or assert (emit an error/ADD_FAILURE/abort) instead of
silently overwriting; update the loop that iterates modules and use the
duplicate-detection branch to report the colliding module_name and the two
indices so dependency resolution problems are caught early.
- Around line 156-159: scan_precise can return non-module/non-interface units
but the code always pushes an entry that later gets compiled as
CompilationKind::ModuleInterface; change the logic around the modules.push_back
call that uses mod.filename/mod.content/result.module_name/result.modules so
that you only enqueue when scan_precise actually identified a real module
interface (e.g., check a returned flag or that result.module_name is non-empty
or result.modules indicates an interface unit) and skip pushing otherwise;
update the conditional around the push to guard by that check so only true
interface modules are added and later compiled as ModuleInterface.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dd2b73c7-1266-4590-97ea-4ff326247a1d

📥 Commits

Reviewing files that changed from the base of the PR and between 9077c31 and 62aa5dd.

📒 Files selected for processing (3)
  • tests/unit/feature/semantic_tokens_tests.cpp
  • tests/unit/test/tester.cpp
  • tests/unit/test/tester.h
✅ Files skipped from review due to trivial changes (1)
  • tests/unit/feature/semantic_tokens_tests.cpp

Comment thread tests/unit/test/tester.cpp
Comment thread tests/unit/test/tester.cpp
…ster

Clear buffers before Phase 1 remap in compile_driver_with_pch to avoid
prepare_driver's try_emplace silently ignoring preamble-bounded remapping.
Also reuse try_compile() in compile_with_modules and move base_cc1_args
to anonymous namespace.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (2)
tests/unit/test/tester.cpp (2)

160-163: ⚠️ Potential issue | 🟠 Major

Only enqueue true interface modules from scan_precise().

Right now every scanned candidate is pushed, including non-module/non-interface units. Those are later compiled as CompilationKind::ModuleInterface, which can fail incorrectly and pollute dependency indexing.

Suggested fix
         auto result = scan_precise(argv, TestVFS::root(), {}, nullptr, vfs);
-        modules.push_back(
-            {mod.filename, mod.content, result.module_name, std::move(result.modules)});
+        if(result.module_name.empty() || !result.is_interface_unit) {
+            continue;
+        }
+        modules.push_back(
+            {mod.filename, mod.content, result.module_name, std::move(result.modules)});
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/unit/test/tester.cpp` around lines 160 - 163, scan_precise() currently
returns candidates of various kinds but the code always does
modules.push_back(...) which enqueues non-interface units and later forces them
to be compiled as CompilationKind::ModuleInterface; change the push site to only
push when the scan result indicates an actual interface module (e.g. check
result.is_interface or result.module_kind == ModuleInterface/Interface) before
calling modules.push_back({mod.filename, mod.content, result.module_name,
std::move(result.modules)}); this keeps non-module/non-interface results out of
the modules list and prevents incorrect CompilationKind::ModuleInterface
compilation and bad dependency indexing.

165-168: ⚠️ Potential issue | 🟠 Major

Detect duplicate module names instead of overwriting map entries.

name_to_index[modules[i].module_name] = i; silently replaces prior modules with the same name, making dependency resolution unstable and potentially binding imports to the wrong PCM.

Suggested fix
     llvm::StringMap<std::size_t> name_to_index;
     for(std::size_t i = 0; i < modules.size(); ++i) {
-        name_to_index[modules[i].module_name] = i;
+        auto [it, inserted] = name_to_index.try_emplace(modules[i].module_name, i);
+        if(!inserted) {
+            LOG_ERROR("Duplicate module name '{}' in '{}' and '{}'",
+                      modules[i].module_name,
+                      modules[it->second].filename,
+                      modules[i].filename);
+            return false;
+        }
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/unit/test/tester.cpp` around lines 165 - 168, The loop that builds
name_to_index currently overwrites entries for duplicate modules
(name_to_index[modules[i].module_name] = i), which hides duplicate module names;
change the logic in the loop that iterates modules to detect existing keys (use
name_to_index.find or contains on modules[i].module_name) and handle duplicates
explicitly—e.g., record an error/throw/abort or append indices to a vector
instead of replacing the prior entry; update references that expect a single
index (name_to_index) accordingly so dependency resolution uses the
duplicate-aware behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@tests/unit/test/tester.cpp`:
- Around line 160-163: scan_precise() currently returns candidates of various
kinds but the code always does modules.push_back(...) which enqueues
non-interface units and later forces them to be compiled as
CompilationKind::ModuleInterface; change the push site to only push when the
scan result indicates an actual interface module (e.g. check result.is_interface
or result.module_kind == ModuleInterface/Interface) before calling
modules.push_back({mod.filename, mod.content, result.module_name,
std::move(result.modules)}); this keeps non-module/non-interface results out of
the modules list and prevents incorrect CompilationKind::ModuleInterface
compilation and bad dependency indexing.
- Around line 165-168: The loop that builds name_to_index currently overwrites
entries for duplicate modules (name_to_index[modules[i].module_name] = i), which
hides duplicate module names; change the logic in the loop that iterates modules
to detect existing keys (use name_to_index.find or contains on
modules[i].module_name) and handle duplicates explicitly—e.g., record an
error/throw/abort or append indices to a vector instead of replacing the prior
entry; update references that expect a single index (name_to_index) accordingly
so dependency resolution uses the duplicate-aware behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e615a4b3-3459-4675-8886-774a43c1542b

📥 Commits

Reviewing files that changed from the base of the PR and between 62aa5dd and 4fb5a75.

📒 Files selected for processing (2)
  • tests/unit/test/tester.cpp
  • tests/unit/test/tester.h

@16bit-ykiko
16bit-ykiko merged commit 9c89d20 into main Apr 11, 2026
15 checks passed
@16bit-ykiko
16bit-ykiko deleted the feat/tester-compile-with-modules branch April 11, 2026 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant