-
Notifications
You must be signed in to change notification settings - Fork 31k
Add SmolLM3 #38755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SmolLM3 #38755
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces the new SmolLM3 modular architecture, adding model implementations, configuration, integration with the auto-model system, tests, and documentation.
- Added SmolLM3 configuration and model modules in src/transformers/models/smollm3
- Extended auto-model routing to support SmolLM3 in the auto configuration and modeling modules
- Introduced a dedicated test suite and documentation for SmolLM3
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| utils/check_config_attributes.py | Added config attribute for SmolLM3Config |
| tests/models/smollm3/test_modeling_smollm3.py | New test suite for SmolLM3 functionality |
| src/transformers/models/smollm3/modular_smollm3.py | New implementation modules for SmolLM3 including its attention layer |
| src/transformers/models/smollm3/configuration_smollm3.py | New configuration for SmolLM3, auto-generated from the modular file |
| src/transformers/models/smollm3/init.py | Updated module export for SmolLM3 |
| src/transformers/models/auto/modeling_auto.py | Registered SmolLM3 model classes in the auto-model system |
| src/transformers/models/auto/configuration_auto.py | Registered SmolLM3 configuration in the auto-model system |
| docs/source/en/model_doc/smollm3.md | Documentation page for the SmolLM3 model |
| docs/source/en/_toctree.yml | Added SmolLM3 to the documentation tree |
Comments suppressed due to low confidence (2)
docs/source/en/model_doc/smollm3.md:53
- [nitpick] The code snippet accesses 'generated_text' with a list index and then a 'content' key, which may be unclear. Please clarify the expected structure of the pipeline output in the documentation or adjust the snippet to match the actual return format.
print(outputs[0]["generated_text"][-1]['content'])
src/transformers/models/smollm3/configuration_smollm3.py:1
- [nitpick] Consider simplifying or removing the repetitive alarm emojis in the header to maintain a professional appearance in the auto-generated file.
# 🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well done! Only TODO for the readme the rest is neat!
Co-authored-by: Arthur <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you just need to run the creation of the modeling file from moudlar to get the CIs green!
What does this PR do?
Adds the SmolLM3 modular arch
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?