Skip to content

Conversation

@anton-l
Copy link
Member

@anton-l anton-l commented Jun 11, 2025

What does this PR do?

Adds the SmolLM3 modular arch

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@anton-l anton-l requested a review from Copilot June 16, 2025 16:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces the new SmolLM3 modular architecture, adding model implementations, configuration, integration with the auto-model system, tests, and documentation.

  • Added SmolLM3 configuration and model modules in src/transformers/models/smollm3
  • Extended auto-model routing to support SmolLM3 in the auto configuration and modeling modules
  • Introduced a dedicated test suite and documentation for SmolLM3

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
utils/check_config_attributes.py Added config attribute for SmolLM3Config
tests/models/smollm3/test_modeling_smollm3.py New test suite for SmolLM3 functionality
src/transformers/models/smollm3/modular_smollm3.py New implementation modules for SmolLM3 including its attention layer
src/transformers/models/smollm3/configuration_smollm3.py New configuration for SmolLM3, auto-generated from the modular file
src/transformers/models/smollm3/init.py Updated module export for SmolLM3
src/transformers/models/auto/modeling_auto.py Registered SmolLM3 model classes in the auto-model system
src/transformers/models/auto/configuration_auto.py Registered SmolLM3 configuration in the auto-model system
docs/source/en/model_doc/smollm3.md Documentation page for the SmolLM3 model
docs/source/en/_toctree.yml Added SmolLM3 to the documentation tree
Comments suppressed due to low confidence (2)

docs/source/en/model_doc/smollm3.md:53

  • [nitpick] The code snippet accesses 'generated_text' with a list index and then a 'content' key, which may be unclear. Please clarify the expected structure of the pipeline output in the documentation or adjust the snippet to match the actual return format.
print(outputs[0]["generated_text"][-1]['content'])

src/transformers/models/smollm3/configuration_smollm3.py:1

  • [nitpick] Consider simplifying or removing the repetitive alarm emojis in the header to maintain a professional appearance in the auto-generated file.
#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨

@anton-l anton-l marked this pull request as ready for review June 21, 2025 13:31
@anton-l anton-l changed the title [WIP] Add SmolLM3 Add SmolLM3 Jun 21, 2025
Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! Only TODO for the readme the rest is neat!

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you just need to run the creation of the modeling file from moudlar to get the CIs green!

@ArthurZucker ArthurZucker enabled auto-merge (squash) June 25, 2025 13:32
@ArthurZucker ArthurZucker merged commit dad0e87 into main Jun 25, 2025
21 checks passed
@ArthurZucker ArthurZucker deleted the add-smollm3 branch June 25, 2025 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants