chore: revive semantic similarity test

[mckornfield]

Summary

Pre-Review Checklist

Ensure that the following pass:

• mise run format && mise run check or via prek validation.
• mise run test passes locally
• mise run test:e2e passes locally
• mise run test:ci-container passes locally (recommended)
• GPU CI status check passes -- comment /sync on this PR to trigger a run (auto-triggers on ready-for-review)

Pre-Merge Checklist

• New or updated tests for any fix or new behavior
• Updated documentation for new features and behaviors, including docstrings for API docs.

Other Notes

• Closes chore: Revisit test_multimodal_report #74

Summary by CodeRabbit

• Tests

  • Improved validation of evaluation report rendering to ensure expected metric labels are present.
  • Made report assembly tests deterministic and more resilient when optional inference dependencies are not available.

• Refactor

  • Lazy-loaded the sentence-transformer dependency to reduce runtime import issues and improve startup robustness.

[coderabbitai]

Walkthrough

Defers SentenceTransformer import to runtime initialization with TYPE_CHECKING for static types, stubs TextSemanticSimilarity in multimodal report tests to avoid dependency, and adds e2e assertions that the evaluation report HTML includes “Synthetic Quality Score” and “Text Semantic Similarity”.

Changes

Lazy-loaded TextSemanticSimilarity and test adaptation

Layer / File(s)	Summary
Lazy import pattern for TextSemanticSimilarity src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py	Top-level SentenceTransformer import removed; TYPE_CHECKING used for type-only import. SentenceTransformer is now imported locally inside _init_sentence_transformer_model with ImportError handling.
Multimodal report test mocking and updates tests/evaluation/reports/test_multimodal_report.py	Replaces module-level pytest.importorskip("sentence_transformers") with an autouse fixture that monkeypatches TextSemanticSimilarity.from_evaluation_datasets to return deterministic scores; updates fixtures/call layout, removes pytest.mark.slow, and adjusts assertions to match stubbed values while loosening SQS numeric checks.
E2E evaluation report rendering validation tests/e2e/test_safe_synthesizer.py	Adds _assert_evaluation_report_rendered(report_html) and invokes it in test_train_and_generate_dp and test_train_and_generate_defaults to assert result.evaluation_report_html is present and contains expected metric labels.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• NVIDIA-NeMo/Safe-Synthesizer#521: Touches the same text_semantic_similarity.py initialization/import handling around SentenceTransformer.

Suggested reviewers

• kendrickb-nvidia
• binaryaaron

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore: revive semantic similarity test' accurately reflects the main objective of the PR, which is to revive the semantic similarity test by refactoring imports, adding test fixtures, and updating test assertions.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch multimodal-test-revisit/mck

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Greptile Summary

This PR makes sentence_transformers a soft/optional dependency by moving its import behind TYPE_CHECKING and adding an ImportError guard in _init_sentence_transformer_model. The test suite is revived to run independently of the package.

• text_semantic_similarity.py: Top-level sentence_transformers import replaced with a lazy in-function import guarded by try/except ImportError, returning None if the package is absent. The type annotation is preserved via TYPE_CHECKING.
• test_multimodal_report.py: importorskip guard removed; an autouse monkeypatch fixture stubs TextSemanticSimilarity at the module level so tests run without model inference. The @pytest.mark.slow mark is also removed and the large-fixture variants are swapped for smaller ones.
• test_safe_synthesizer.py: A shared helper _assert_evaluation_report_rendered validates that the report HTML contains expected metric-label strings in both e2e test cases.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to making an optional dependency truly optional, with no effect on core pipeline behavior when the package is present.

The source change is additive and backward-compatible: when sentence_transformers is installed everything behaves exactly as before, and when it is absent the code now degrades gracefully instead of crashing at import time. The test changes replace a coarse importorskip guard with a precise monkeypatch stub, improving isolation without weakening coverage. No logic paths are removed or bypassed.

No files require special attention.

Important Files Changed

Filename	Overview
src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py	Moves sentence_transformers import to TYPE_CHECKING and adds ImportError guard in _init_sentence_transformer_model, making the dependency optional.
tests/evaluation/reports/test_multimodal_report.py	Replaces importorskip with an autouse monkeypatch stub, removes slow marks and large fixtures; assertions updated to check Text Semantic Similarity values from the stub.
tests/e2e/test_safe_synthesizer.py	Adds _assert_evaluation_report_rendered helper to validate HTML report content in e2e tests, no logic issues.

Sequence Diagram

sequenceDiagram
    participant Test as Test Suite
    participant Stub as StubTextSemanticSimilarity
    participant MMR as MultimodalReport
    participant TSS as TextSemanticSimilarity
    participant ST as sentence_transformers

    Note over Test,ST: Without sentence_transformers installed (unit tests)
    Test->>MMR: from_dataframes(...)
    MMR->>Stub: from_evaluation_datasets(...) [monkeypatched]
    Stub-->>MMR: "TextSemanticSimilarity(score=9.0)"
    MMR-->>Test: report (no model inference)

    Note over Test,ST: With sentence_transformers installed (e2e/GPU tests)
    Test->>MMR: from_dataframes(...)
    MMR->>TSS: from_evaluation_datasets(...)
    TSS->>TSS: _init_sentence_transformer_model()
    alt sentence_transformers available
        TSS->>ST: import SentenceTransformer
        ST-->>TSS: model loaded
        TSS-->>MMR: "TextSemanticSimilarity(score=computed)"
    else ImportError
        TSS-->>MMR: "TextSemanticSimilarity(score=null)"
    end
    MMR-->>Test: report

Loading

_{Reviews (2): Last reviewed commit: "chore: revive semantic similarity test" | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 67.85714% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
tests/e2e/test_safe_synthesizer.py	16.66%	5 Missing ⚠️
.../evaluation/components/text_semantic_similarity.py	20.00%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py (1)

243-257: ⚠️ Potential issue | 🟠 Major

Fail fast when sentence_transformers is unavailable instead of retrying the import.

In TextSemanticSimilarity._init_sentence_transformer_model(), from sentence_transformers import SentenceTransformer runs inside the Retrying(...) block (no custom retry= filter). Tenacity’s default retries on any Exception, so a missing dependency (ImportError) backs off up to ~367s before returning None; the caller then raises when stm is None, meaning text semantic similarity reporting can stall and then fail for the component (and it’s called once per text_fields entry).

Smallest fix: import SentenceTransformer outside the retry loop (or configure Retrying(retry=...) to exclude ImportError) so only model construction/loading failures are retried.

Source: Coding guidelines

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e3823f7d-7de7-49e2-b361-4fbcb53aa7ef

📥 Commits

Reviewing files that changed from the base of the PR and between 72ecdd3 and e6af7d3.

📒 Files selected for processing (3)

• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

📜 Review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)

• GitHub Check: Unit Tests (3.11)
• GitHub Check: Unit Tests (3.13)
• GitHub Check: Unit Tests (3.12)
• GitHub Check: Smoke Tests
• GitHub Check: Typecheck
• GitHub Check: Analyze (Python)
• GitHub Check: Analyze (python)

🧰 Additional context used

📓 Path-based instructions (12)

**/*.{md,markdown,py}

📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)

**/*.{md,markdown,py}: Avoid decorative bold (**text**) in list items, body text, and docstrings; use structural cues (headers, list markers, colons, backticks) for emphasis instead
Use backticks for code identifiers, paths, and CLI commands in markdown and docstrings

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Place durable implementation guidance in function and class docstrings for public contracts and source comments for local invariants
Target Python 3.11–3.13 with modern syntax (X | Y, list[str], Self). Python 3.14+ is not supported

**/*.py: Python source files must use ruff for formatting and linting. Run mise run format to auto-fix formatting and linting issues, and mise run check for read-only quality checks
Use ty for Python type checking. Run mise run check to execute type checking on all tracked files

**/*.py: Use Field(description=...) as the canonical field docstring for Pydantic models.
Use assignment style (type = Field(default=..., description="...")) as the default for Pydantic model fields rather than Annotated-only patterns.
Use Annotated only when the field carries additional metadata beyond Field() -- ValueValidator, AutoParam, DependsOnValidator, reusable constrained type aliases, nested-type constraints, or discriminated unions.
Use @dataclass(frozen=True) for immutable value objects and validators. Mutable @dataclass acceptable for builders, accumulators, and pipeline state.
Use field(default_factory=list) for mutable defaults in dataclasses, never = [].
Use StrEnum for string-valued enums used in configs/serialization. Plain Enum for internal-only named constants.
Use X | Y not Optional[X] or Union[X, Y] in type hints.
Use list[str] not List[str], dict[str, int] not Dict[str, int] in type hints.
Use Self for fluent method returns in type hints.
Use collection ABCs for function arguments (Sequence, Mapping, Iterable) so callers can pass any compatible container; concrete types for return values.
Use Protocol for structural subtyping when you need duck-typing boundaries.
Avoid Any in type hints -- prefer object, generics, or Protocol.
Use TYPE_CHECKING guards for heavy imports (pandas, torch, transformers); not needed for stdlib or lightweight imports.
Prefer `m...

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

**/test_*.py

📄 CodeRabbit inference engine (AGENTS.md)

Use the unit marker instead of the deprecated unit_test marker for test identification

Files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

tests/**

📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)

tests/**: Mirror src/ directory structure in tests/ directory for test organization
Auto-mark tests by directory: tests/e2e/ → e2e, tests/smoke/ → smoke, otherwise default to unit

Files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

---

⚙️ CodeRabbit configuration file

tests/**:

Testing Guide

Comprehensive testing reference for Safe-Synthesizer developers. Covers commands, markers, test data, fixtures, and gotchas.

Read First

1. tests/conftest.py -- auto-marking, load_test_dataset/load_test_dataframe, fixture_mock_processor pattern
2. pytest.ini -- markers, asyncio, timeout
3. tests/evaluation/conftest.py -- most complex: Faker-based make_df, nullable dtype conversion
4. tests/generation/conftest.py -- JSONL/schema fixtures, fixture_valid_iris_dataset_jsonl_and_schema

Running Tests

All mise test tasks, grouped by scope:

mise run test                              # Unit (excludes slow, e2e, and smoke)
mise run test:unit-slow                    # Unit tests including slow (excludes e2e and smoke)
mise run test:smoke                        # CPU smoke tests (~few min, no GPU required)
mise run test:smoke:gpu                    # All staged GPU smoke tests (requires CUDA)
mise run test:smoke:gpu:train-only
mise run test:smoke:gpu:generation
mise run test:smoke:gpu:resume
mise run test:smoke:gpu:structured-generation
mise run test:smoke:gpu:timeseries
mise run test:smoke:gpu:smollm2
mise run test:e2e                          # All e2e (requires CUDA) -- runs default + dp
mise run test:e2e:default                  # e2e default (no-DP) tests only
mise run test:e2e:dp                       # e2e DP tests only
mise run test:ci                           # CI unit tests with coverage (excludes slow, e2e, gpu, smoke)
mise run test:ci-slow                      # CI slow tests with coverage
mise run test:ci-container                 # CI tests in a Linux container (Docker/Podman)

Run a single test:

uv run --frozen pytest tests/path/test_file.py::test_name -vvs -n0

Test runner: uv run --frozen pytest -n auto --dist loadscope -vv...

Files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

**/*.{py,sh,yaml,yml,md}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

All source files (.py, .sh, .yaml, .yml, .md) must include SPDX copyright headers. Use mise run format to add them automatically

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

tests/**/*.py

📄 CodeRabbit inference engine (CONTRIBUTING.md)

tests/**/*.py: New features must include tests before submitting a PR
Bug fixes must include regression tests before submitting a PR

tests/**/*.py: Use file naming test_*.py, class naming Test*, function naming test_<module>_<expected_behavior> for test files.
Use function-scoped fixtures by default. Session scope only when empirically justified by test runtime.
Use bare assert as primary assertion style; pytest.raises() with match= for exceptions; pytest.approx() for floating-point comparisons.
Docstrings are optional for simple tests, recommended for complex/e2e tests explaining purpose.
Markers are auto-assigned by path via pytest_collection_modifyitems (/e2e/ -> e2e, /smoke/ -> smoke, default -> unit). Use explicit markers: @pytest.mark.slow, @pytest.mark.requires_gpu, @pytest.mark.timeout().
Use tmp_path fixture for file operations in tests, never write to the repo tree.
Mark CUDA-dependent tests with @pytest.mark.e2e, @pytest.mark.smoke, or @pytest.mark.requires_gpu.
Mock only external boundaries in tests, not internal implementation details.
Ensure test isolation: no shared mutable state or execution-order dependencies between tests. Include required setup in the test or a fixture.
Use @pytest.mark.parametrize for testing multiple input combinations rather than copy-pasting similar tests.

tests/**/*.py: Assign exactly one of three pytest category markers (unit, smoke, e2e) to every test function
Markers 'slow' and 'requires_gpu' modify the three category markers (unit, smoke, e2e) to indicate when tests should be run or if separate pytest invocations are required
Use 'pytest.mark.noautouse' to skip autouse fixtures for specific tests
Auto-marking via pytest_collection_modifyitems assigns category markers based on test file path: '/e2e/' → 'e2e', '/smoke/' → 'smoke', no match → 'unit', only if no category marker is already present
Use load_test_dataset(filename) from root conftest.py to load HuggingFace Dataset...

Files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

---

⚙️ CodeRabbit configuration file

Review tests against tests/TESTING.md. Check marker usage, fixture naming, tmp_path usage, determinism, and GPU/vLLM process-isolation requirements. Flag slop tests that only check that code runs, assert result is not None when stronger invariants exist, over-mock internal implementation details, patch around the bug instead of reproducing it, or add broad snapshot/golden churn without a clear contract. Flag change detector tests that fail on harmless refactors, formatting, record ordering, incidental wording, or private implementation details without demonstrating a behavior regression. Prefer existing fixtures or focused new fixtures for repeated setup; keep tests DRY when reasonable without making the behavior under test opaque. print() is allowed in tests.

Files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

**/*.{py,sh,yaml,yml}

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Include SPDX copyright headers in all source files using hash-comments for .py, .sh, .yaml, .yml files.

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

**/*

⚙️ CodeRabbit configuration file

**/*: Review as a senior maintainer for NeMo Safe Synthesizer. Prioritize issues that can change behavior, break user workflows, weaken privacy guarantees, hide failures, make tests unreliable, or create maintenance risk. Avoid generic style commentary unless it points to a concrete project convention that automated tools will not catch.
Comment only when the finding is actionable and tied to changed code. For each finding, state the impact, the condition that triggers it, and the smallest practical fix. Prefer one precise comment over broad advice. Do not ask for refactors outside the PR scope unless the changed code creates the problem.
Review type guidance: - Potential issue: use for correctness bugs, data loss, privacy leaks,
security risks, broken public APIs, invalid config behavior, missing
validation, hidden failures, nondeterministic tests, or CI breakage.

• Refactor suggestion: use for local maintainability problems introduced
  by the diff when they have clear future cost, such as duplicated setup,
  unclear boundaries, over-mocking, avoidable complexity, or opaque test
  helpers.
• Nitpick: avoid in chill mode. Do not emit formatting, import-order,
  wording, or style-only comments unless automated tools cannot catch the
  issue and it affects maintainability.

Severity guidance: - Critical: security/privacy leaks, data loss, training/test/holdout
contamination, or broken release/package/core pipeline execution.

• Major: incorrect generation/training/evaluation behavior, broken
  CLI/SDK public API, invalid config defaults or validators, or GPU/vLLM
  cleanup and process-isolation bugs likely to fail CI or production
  runs.
• Minor: localized bugs, missing focused tests for changed behavior, or
  bad test patterns that weaken regression coverage.
• Trivial: small cleanup with no behavior impact. Usually suppress in
  chill mode.
• Info: context only. Avoid unless it helps reviewers understand risk.
  Safe-Synthesizer-specific review focus: - Data ...

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

**

⚙️ CodeRabbit configuration file

**:

AGENTS.md

Guide for AI agents (Cursor, Windsurf, Claude Code, etc.) working in the Safe-Synthesizer repo.

This project loads local developer preferences from @AGENTS.local.md. You MUST read this file if it exists and give its instructions top priority.

Skills

Repo-specific skills live in .agents/skills/; see .agents/README.md for the catalog. Read a skill when the task matches its scope instead of copying workflow details into this file.

Durable implementation guidance belongs with the code it describes: function and class docstrings for public contracts and source comments for local invariants. Test-suite guidance belongs in tests/TESTING.md.

Repo Conventions

See STYLE_GUIDE.md for detailed code style conventions (Python, markdown, Dockerfiles, shell scripts, testing, config files, docstrings).

Use uv for everything -- never pip or raw python. Python 3.11–3.13 with modern syntax (X | Y, list[str], Self). Python 3.14+ is not supported.

Common commands: mise run test (unit tests), mise run format (auto-fix formatting + lint + copyright), mise run check (read-only local quality checks), mise run validate (pre-PR quality, lock, and CI unit checks), mise run typecheck (ty only). Always use mise tasks or the wrapper scripts in tools/ instead of running ruff or ty directly. Use uv run for Python execution. When in doubt, inspect mise tasks and pytest --markers.

The canonical uv sync command for a full GPU/dev environment is:

uv sync --frozen --extra cu129 --extra engine --group dev

Bare uv sync --frozen (without extras) installs an incomplete environment -- ty, import checks, and GPU tests will fail.

Feature branches off main. Branch names often include an issue number prefix (e.g., <author>/123-short-name).

Do ...

Files:

• tests/e2e/test_safe_synthesizer.py
• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py
• tests/evaluation/reports/test_multimodal_report.py

src/nemo_safe_synthesizer/**/*.py

📄 CodeRabbit inference engine (CONTRIBUTING.md)

src/nemo_safe_synthesizer/**/*.py: Source code must remain Python 3.11 syntax-compatible. Do not use Python 3.12-only syntax such as PEP 695 type statements or bracketed generic class/function parameters in shared package code
Write Python docstrings in Google style format so they auto-generate into API reference pages via mkdocstrings and gen-files plugins

Files:

• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py

src/**/*.py

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

src/**/*.py: Use observability.get_logger(__name__) -- never logging.getLogger() or structlog.get_logger() directly.
Use category loggers: .runtime for internals, .user for progress/results, .system for system events.
Never use print() for operational output. Use click.echo() for CLI output or sys.stdout.write() for raw output in tools.
Use extra={} for data that downstream tools should query or aggregate in log calls (metrics, counts, durations). f-strings are fine for human-readable context.
Raise from the custom error hierarchy with dual inheritance: SafeSynthesizerError (base), UserError, DataError, ParameterError, GenerationError, InternalError.
Prefer clamping/saturation over raising when out-of-range inputs shouldn't crash the system -- return a bounded value with a log warning.
Error messages must precisely match the actual error condition. Use !r for repr of interpolated pieces to clearly identify them.
Use relative imports in src/ (from ..observability import get_logger), absolute imports in tests/ (from nemo_safe_synthesizer.observability import get_logger).
Use pathlib.Path instead of os.path. Tolerate os.path only in vendored/tooling scripts.
Do not use print() statements in library code. Use get_logger(__name__) from observability.py or click.echo() for CLI. print() is fine in tests, standalone scripts, and tooling.
Do not use assert for validation in library code. Use if/raise for input validation. assert is fine in tests where pytest relies on it.

Files:

• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py

---

⚙️ CodeRabbit configuration file

Review library code against STYLE_GUIDE.md. Focus on behavior, API contracts, error handling, resource cleanup, typing, logging, and user-facing failures. Public APIs and nontrivial functions need Google-style docstrings.

Files:

• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py

src/nemo_safe_synthesizer/evaluation/**/*.py

⚙️ CodeRabbit configuration file

Treat evaluation changes as correctness-sensitive. Check metric inputs, holdout usage, privacy metric semantics, report data shape, missing-data handling, and whether unavailable metrics fail or degrade intentionally.

Files:

• src/nemo_safe_synthesizer/evaluation/components/text_semantic_similarity.py

🧠 Learnings (1)

📚 Learning: 2026-05-27T22:20:37.354Z

Learnt from: kendrickb-nvidia
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 520
File: tests/generation/test_vllm_backend.py:556-587
Timestamp: 2026-05-27T22:20:37.354Z
Learning: In NVIDIA-NeMo/Safe-Synthesizer, `tests/conftest.py`’s `pytest_collection_modifyitems` hook applies pytest category markers automatically based on each test file’s path: tests under `/e2e/` get `pytest.mark.e2e`, tests under `/smoke/` get `pytest.mark.smoke`, and all other tests get `pytest.mark.unit`. Therefore, when reviewing pytest tests outside `tests/e2e/` and `tests/smoke/`, do not flag missing explicit `pytest.mark.unit` decorators on test classes/functions as an issue (the hook will add them during collection). If a new test directory/category is introduced, ensure the hook is updated so it’s categorized correctly.

Applied to files:

• tests/e2e/test_safe_synthesizer.py
• tests/evaluation/reports/test_multimodal_report.py

🔇 Additional comments (1)

tests/e2e/test_safe_synthesizer.py (1)

43-47: LGTM!

Also applies to: 75-75, 99-99