Skip to content

TE FP8 integration#1096

Merged
regisss merged 4 commits into
huggingface:mainfrom
HabanaAI:te-fp8-upstream
Jul 8, 2024
Merged

TE FP8 integration#1096
regisss merged 4 commits into
huggingface:mainfrom
HabanaAI:te-fp8-upstream

Conversation

@scsudhak-intel
Copy link
Copy Markdown
Contributor

@scsudhak-intel scsudhak-intel commented Jun 25, 2024

What does this PR do?

Refactoring transformer_engine FP8 support in optimum-habana. This removes model specific changes and makes the code more model agnostic.

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@scsudhak-intel scsudhak-intel marked this pull request as ready for review June 26, 2024 07:28
@scsudhak-intel scsudhak-intel requested a review from regisss as a code owner June 26, 2024 07:28
@vivekgoe vivekgoe requested a review from libinta June 26, 2024 08:02
Comment thread examples/language-modeling/README.md
Comment thread optimum/habana/transformers/training_args.py
@vivekgoe
Copy link
Copy Markdown
Collaborator

vivekgoe commented Jun 26, 2024

@regisss @libinta please help review this PR. This is a cleaned up version of TE integration (tested with Llama-7B, Llama-70B, Mixtral-7B fine-tuning tasks). Changes were done mainly for following reasons,

  • Align to newer PEFT version (0.10.0) which helps reducing custom changes to support LoRA layers in fp8,
  • Add TE support for models using activation_checkpointing (in a generic way),
  • Align TE code to accelerator repo to extent possible. Limit fp8 configuration interface exposed to user (allow only well tested configuration for now)

FYI @sanjucsudhakaran will create another PR adding Llama-7B test for this feature. It is tested with 1.16.0 release and can be merged without waiting for 1.17.0

@vivekgoe vivekgoe added the run-test Run CI for PRs from external contributors label Jun 26, 2024
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@vivekgoe
Copy link
Copy Markdown
Collaborator

Related PR with tests #1106

@vivekgoe vivekgoe added the synapse1.17 PR that should be available along with Synapse 1.17 but have no dependency on Synapse 1.17 content. label Jun 28, 2024
Comment thread examples/language-modeling/README.md
@vivekgoe vivekgoe added run-test Run CI for PRs from external contributors and removed run-test Run CI for PRs from external contributors labels Jul 8, 2024
@regisss regisss merged commit f3f688b into huggingface:main Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-test Run CI for PRs from external contributors synapse1.17 PR that should be available along with Synapse 1.17 but have no dependency on Synapse 1.17 content.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants