Skip to content

Fix ruff formatting for interleaved module - #1572

Merged
VibhuJawa merged 1 commit into
NVIDIA-NeMo:mainfrom
VibhuJawa:fix/ruff-precommit-cleanup
Mar 4, 2026
Merged

Fix ruff formatting for interleaved module#1572
VibhuJawa merged 1 commit into
NVIDIA-NeMo:mainfrom
VibhuJawa:fix/ruff-precommit-cleanup

Conversation

@VibhuJawa

Copy link
Copy Markdown
Contributor

Summary

Files changed

  • benchmarking/scripts/multimodal_mint1t_benchmark.py
  • nemo_curator/stages/interleaved/io/readers/webdataset.py
  • nemo_curator/tasks/interleaved.py
  • tests/stages/interleaved/test_interleaved_task.py
  • tests/stages/interleaved/test_materialization.py
  • tests/stages/interleaved/test_multimodal_core.py
  • tests/stages/interleaved/test_multimodal_reader.py
  • tests/stages/interleaved/test_multimodal_writer.py
  • tutorials/multimodal/mint1t_mvp_pipeline.py

Test plan

  • All pre-commit hooks pass (ruff, ruff format, trailing whitespace, etc.)
  • CI passes (no logic changes, formatting only)

Made with Cursor

…1517

Apply ruff format auto-fixes from pre-commit hooks to 9 files
added/modified in the multimodal reader/writer PR.

Signed-off-by: Vibhu Jawa <vjawa@nvidia.com>
Made-with: Cursor
@greptile-apps

greptile-apps Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR applies ruff format auto-fixes to 9 files introduced/modified in #1517, ensuring the interleaved multimodal module passes pre-commit hooks. The changes are purely cosmetic, with no logic or behavioral differences.

  • Line wrapping: Long single-line function calls (e.g., pipeline.add_stage(InterleavedAspectRatioFilterStage(...)), method signatures) are split across multiple lines to comply with the project's 119-character line length limit.
  • Dict/argument expansion: Inline dictionary literals and constructor arguments (e.g., pd.DataFrame({...}), InterleavedBatch(task_id=..., dataset_name=...)) are reformatted to one key-value or argument per line.
  • Collapsing short expressions: Two _image_task([_image_row(...)]) calls in test_multimodal_core.py are collapsed from multi-line to single line, as both fit within 119 characters — this is consistent with ruff format's magic trailing comma behavior.
  • String concatenation: One multi-line implicit string concatenation (f"..." f"...") in webdataset.py is collapsed to a single f-string (fits within 119 chars).

Confidence Score: 5/5

  • This PR is safe to merge — all changes are mechanical ruff format auto-fixes with no logic modifications.
  • Every change is a pure formatting transformation (line wrapping, argument expansion, trailing comma addition, or short-expression collapsing) produced by ruff format against the project's configured 119-character line length. No conditional logic, data flow, imports, or API signatures were altered.
  • No files require special attention.

Important Files Changed

Filename Overview
benchmarking/scripts/multimodal_mint1t_benchmark.py Single formatting change: wraps pipeline.add_stage(InterleavedAspectRatioFilterStage(...)) onto multiple lines to comply with 119-char line limit. No logic changes.
nemo_curator/stages/interleaved/io/readers/webdataset.py Several formatting changes: method signatures expanded to one-argument-per-line, inline dict literals reformatted as multi-line, and one multi-line error message string collapsed to a single line (fits within 119 chars). No logic changes.
nemo_curator/tasks/interleaved.py Single formatting change: inline ref dict in build_source_ref expanded to one key-value pair per line. No logic changes.
tests/stages/interleaved/test_interleaved_task.py Single formatting change: InterleavedBatch.build_source_ref(...) call arguments expanded to one per line. No logic changes.
tests/stages/interleaved/test_materialization.py Multiple formatting changes: pd.DataFrame({...}) dict literals and InterleavedBatch(task_id=..., dataset_name=...) constructor calls expanded to multi-line. No logic changes.
tests/stages/interleaved/test_multimodal_core.py Extensive formatting changes across many test functions: inline row dicts expanded to multi-line, and two _image_task([...]) calls collapsed from multi-line to single line (both fit within 119-char limit). No logic changes.
tests/stages/interleaved/test_multimodal_reader.py Multiple formatting changes: WebdatasetReaderStage(...) and FileGroupTask(...) constructor arguments expanded to one per line. No logic changes.
tests/stages/interleaved/test_multimodal_writer.py Multiple formatting changes: pd.DataFrame([{...}]) and pa.Table.from_pylist([{...}]) calls expanded to multi-line, InterleavedParquetWriterStage(...) arguments split to one per line. No logic changes.
tutorials/multimodal/mint1t_mvp_pipeline.py Single formatting change: wraps pipe.add_stage(InterleavedAspectRatioFilterStage(...)) onto multiple lines. No logic changes.

Sequence Diagram

sequenceDiagram
    participant Pipeline
    participant WebdatasetReaderStage
    participant InterleavedBatch
    participant InterleavedAspectRatioFilterStage
    participant InterleavedParquetWriterStage

    Pipeline->>WebdatasetReaderStage: process(FileGroupTask)
    WebdatasetReaderStage->>WebdatasetReaderStage: _build_source_ref(ctx, content_key)
    WebdatasetReaderStage->>InterleavedBatch: build_source_ref(path, member, byte_offset, byte_size)
    WebdatasetReaderStage-->>Pipeline: InterleavedBatch (rows)

    Pipeline->>InterleavedAspectRatioFilterStage: process(InterleavedBatch)
    InterleavedAspectRatioFilterStage-->>Pipeline: filtered InterleavedBatch

    Pipeline->>InterleavedParquetWriterStage: process(InterleavedBatch)
    InterleavedParquetWriterStage-->>Pipeline: written FileGroupTask
Loading

Last reviewed commit: 64b0f3c

@VibhuJawa
VibhuJawa enabled auto-merge (squash) March 4, 2026 21:34
@VibhuJawa
VibhuJawa merged commit d378727 into NVIDIA-NeMo:main Mar 4, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants