Skip to content

update paper link, references to dataset, self-correction differences#1129

Merged
stephencge merged 1 commit intomainfrom
docs/nemotron-math-proofs-updates
Dec 18, 2025
Merged

update paper link, references to dataset, self-correction differences#1129
stephencge merged 1 commit intomainfrom
docs/nemotron-math-proofs-updates

Conversation

@stephencge
Copy link
Collaborator

@stephencge stephencge commented Dec 18, 2025

Summary by CodeRabbit

  • Documentation
    • Updated release notes with clearer explanations of dataset filtering criteria and evaluation methodology.
    • Enhanced descriptions of model training and theorem proving processes with additional context and references.
    • Added clarifying annotations in results tables explaining evaluation configuration and model behavior differences.

✏️ Tip: You can customize this high-level summary in your review settings.

Signed-off-by: Stephen Ge <stepheng@nvidia.com>
@stephencge stephencge requested a review from Kipok December 18, 2025 15:00
@stephencge stephencge enabled auto-merge (squash) December 18, 2025 15:02
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 18, 2025

📝 Walkthrough

Walkthrough

Documentation updates to the Nemotron Math Proofs release page, including rewording the Natural Language Problems section with filtering criteria, adding paper references, annotating training results with footnotes, expanding sections with evaluation methodology details, clarifying dataset field descriptions, and updating CLI/Python sections with dataset linkage.

Changes

Cohort / File(s) Summary
Documentation Updates
docs/releases/nemotronmathproofs/index.md
Rewording Natural Language Problems section with explicit filtering criteria; added paper references; reformatted training results table with pass@32 annotations; expanded sections with self-correction attempt details; clarified input field descriptions; updated CLI/Python sections with dataset field references

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

  • Documentation-only changes affecting a single file
  • Primary activities: content rewording, clarifications, and contextual additions
  • No code logic or functional changes to review

Possibly related PRs

Suggested reviewers

  • gwarmstrong
  • Kipok

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: updating paper link references, dataset references, and documenting self-correction differences in the documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch docs/nemotron-math-proofs-updates

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1cac9db and 6027d0b.

📒 Files selected for processing (1)
  • docs/releases/nemotronmathproofs/index.md (5 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: pre-commit
🔇 Additional comments (3)
docs/releases/nemotronmathproofs/index.md (3)

33-42: Well-executed table annotations with clear footnote explanations.

The footnotes effectively explain the self-correction differences between models and pipeline limitations. The subsequent narrative block (lines 41-42) provides good context for the second table.


1-382: Comprehensive documentation update with clear methodology explanations.

The changes effectively communicate dataset composition, training results, and reproduction workflows. The addition of footnotes explaining self-correction differences and field references linking to dataset documentation improves clarity.


Summary

Your documentation updates look solid overall. The main items needing verification are:

  1. Paper link validity (line 19): The arXiv link format appears standard but should be confirmed as published/accessible.

  2. Dataset field names (lines 62-64, 138-139, 210-211): The referenced fields (problem, formal_statement, lean_header, messages) should be verified against the actual dataset schema to prevent user confusion if field names differ.

  3. External links: All HuggingFace dataset links should resolve correctly.

The structural and informational changes—footnotes explaining self-correction differences, narrative blocks clarifying evaluation methodology, and field-level documentation—are well-executed and align with the PR objectives.


16-19: Verify arXiv paper link accessibility and correctness.

The documentation references an arXiv paper at https://arxiv.org/abs/2512.15489 for details on problem sources and extraction pipeline. The paper ID could not be verified as published or accessible. Confirm this link is correct and live before publication, or provide alternative documentation if the paper is not yet published.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@stephencge stephencge merged commit eed4f7e into main Dec 18, 2025
5 of 6 checks passed
@stephencge stephencge deleted the docs/nemotron-math-proofs-updates branch December 18, 2025 17:25
wasiahmad pushed a commit that referenced this pull request Dec 19, 2025
wasiahmad pushed a commit that referenced this pull request Dec 19, 2025
…#1129)

Signed-off-by: Stephen Ge <stepheng@nvidia.com>

Signed-off-by: wasiahmad <wasiahmad@ucla.edu>
blahblahasdf pushed a commit to blahblahasdf/Skills that referenced this pull request Jan 8, 2026
…NVIDIA-NeMo#1129)

Signed-off-by: Stephen Ge <stepheng@nvidia.com>
Signed-off-by: dlord <dlord@nvidia.com>
hsiehjackson pushed a commit that referenced this pull request Jan 13, 2026
…#1129)

Signed-off-by: Stephen Ge <stepheng@nvidia.com>
Signed-off-by: Cheng-Ping Hsieh <chsieh@nvidia.com>
wasiahmad pushed a commit that referenced this pull request Feb 4, 2026
dgtm777 pushed a commit that referenced this pull request Mar 18, 2026
dgtm777 pushed a commit that referenced this pull request Mar 18, 2026
…#1129)

Signed-off-by: Stephen Ge <stepheng@nvidia.com>
Signed-off-by: dgitman <dgitman@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants