Skip to content

Add configurable DeepSpeed installation via DEEPSPEED_SPEC env var in Makefile#2229

Merged
regisss merged 2 commits into
huggingface:mainfrom
HabanaAI:main-custom-deepspeed-install
Aug 27, 2025
Merged

Add configurable DeepSpeed installation via DEEPSPEED_SPEC env var in Makefile#2229
regisss merged 2 commits into
huggingface:mainfrom
HabanaAI:main-custom-deepspeed-install

Conversation

@gplutop7
Copy link
Copy Markdown
Collaborator

This PR updates the Makefile so that DeepSpeed installation is no longer hardcoded.
Instead, the version (or fork) to install can now be customized through the environment variable DEEPSPEED_SPEC.

  • The default value of DEEPSPEED_SPEC is defined directly in the Makefile (currently pointing to the HabanaAI repository and version).
  • Users can override it by exporting their own value before running the tests.

Examples:

Default behavior (uses the value defined in the Makefile):
make slow_tests_text_generation_example_8x

Custom behavior (installs from internal fork with a GitHub token):

export DEEPSPEED_SPEC="git+https://${GH_TOKEN}@github.com/habana-internal/deepspeed-fork.git@v1.22.0"
make slow_tests_text_generation_example_8x

This makes it easier to switch between official and internal DeepSpeed forks or test new versions without modifying the Makefile itself.

@gplutop7 gplutop7 requested a review from regisss as a code owner August 27, 2025 11:19
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment thread Makefile Outdated
Copy link
Copy Markdown
Collaborator

@regisss regisss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@regisss regisss merged commit 9c76ca4 into huggingface:main Aug 27, 2025
2 of 4 checks passed
imangohari1 pushed a commit to imangohari1/optimum-habana that referenced this pull request Aug 28, 2025
gplutop7 pushed a commit to HabanaAI/optimum-habana-fork that referenced this pull request Oct 15, 2025
… Makefile (huggingface#2229) (huggingface#635)

Co-authored-by: Grzegorz Pluto-Prondzinski <gplutopx@habana.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants