Skip to content

Mark scale as const and remove --fp8 flag usage #962

Merged
regisss merged 10 commits into
huggingface:mainfrom
HabanaAI:dev/yan_pr
May 28, 2024
Merged

Mark scale as const and remove --fp8 flag usage #962
regisss merged 10 commits into
huggingface:mainfrom
HabanaAI:dev/yan_pr

Conversation

@Yantom1
Copy link
Copy Markdown
Contributor

@Yantom1 Yantom1 commented May 8, 2024

  • Mark only scales as const

  • remove --fp8 flag usage from llama

  • removed usage of ENABLE_CONST_MARKING

Change-Id: I6dba8691d842fc62d09da5202ea1e61a111f5f18

What does this PR do?

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@Yantom1
Mark scale as const and remove --fp8 flag usage (#156)
6e4266e 
* Mark only scales as const

* remove --fp8 flag usage from llama

* removed usage of ENABLE_CONST_MARKING

Change-Id: I6dba8691d842fc62d09da5202ea1e61a111f5f18

---------

Co-authored-by: Eran Geva <egeva@habana.ai>
@Yantom1 Yantom1 requested a review from regisss as a code owner May 8, 2024 07:53
@regisss
Copy link
Copy Markdown
Collaborator

regisss commented May 8, 2024

Is this linked to #966 ?

@Yantom1
Copy link
Copy Markdown
Contributor Author

Yantom1 commented May 8, 2024

regisss yes its connected

@libinta libinta added the synapse 1.16_dependency synapse 1.16 dependency label May 8, 2024
ssarkar2
ssarkar2 reviewed May 8, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@ssarkar2 ssarkar2 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Can you put the changes of #966 into this PR, then it will be easy to merge together, because both are related.

  2. Can you modify this test, because your chaneg removes fp8: https://github.com/huggingface/optimum-habana/blob/3a14236643916ecc5112f23560874cca083f2044/tests/test_text_generation_example.py
    esp these lines:

    optimum-habana/tests/test_text_generation_example.py

    Line 182 in 3a14236

    command.insert(-2, "--fp8")

@Yantom1
Copy link
Copy Markdown
Contributor Author

Yantom1 commented May 12, 2024 •
edited
Loading

  1. Can you put the changes of Remove --fp8 flag from README #966 into this PR, then it will be easy to merge together, because both are related.
  2. Can you modify this test, because your chaneg removes fp8: https://github.com/huggingface/optimum-habana/blob/3a14236643916ecc5112f23560874cca083f2044/tests/test_text_generation_example.py
    esp these lines:

    optimum-habana/tests/test_text_generation_example.py

    Line 182 in 3a14236

    command.insert(-2, "--fp8")

Hey @ssarkar2 , both fixed.

@regisss
Copy link
Copy Markdown
Collaborator

regisss commented May 13, 2024

@Yantom1 It seems there are some merge conflicts, can you merge main into your branch and fix them please? Let me know if there is anything you're not sure about.

@Yantom1
Copy link
Copy Markdown
Contributor Author

Yantom1 commented May 13, 2024

@Yantom1 It seems there are some merge conflicts, can you merge main into your branch and fix them please? Let me know if there is anything you're not sure about.

@regisss done

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

HuggingFaceDocBuilderDev commented May 16, 2024

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@libinta libinta added run-test Run CI for PRs from external contributors synapse1.16 and removed synapse 1.16_dependency synapse 1.16 dependency labels May 16, 2024
regisss
regisss reviewed May 20, 2024
View reviewed changes
Comment thread examples/text-generation/README.md
regisss
regisss reviewed May 23, 2024
View reviewed changes
Comment thread examples/text-generation/README.md
Comment thread examples/text-generation/README.md
regisss
regisss reviewed May 24, 2024
View reviewed changes
Comment thread examples/text-generation/README.md Outdated
Comment thread examples/text-generation/README.md Outdated
Comment thread examples/text-generation/README.md Outdated
Comment thread examples/text-generation/README.md Outdated
@Yantom1
Copy link
Copy Markdown
Contributor Author

Yantom1 commented May 24, 2024

@regisss fixed the last comments also

@regisss regisss added run-test Run CI for PRs from external contributors and removed run-test Run CI for PRs from external contributors labels May 24, 2024
@regisss regisss merged commit 3e5e02d into huggingface:main May 28, 2024
@Yantom1 Yantom1 mentioned this pull request Jun 11, 2024
Merged
3 tasks
@astachowiczhabana astachowiczhabana mentioned this pull request Jun 12, 2024
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@regisss regisss regisss approved these changes

+1 more reviewer

@ssarkar2 ssarkar2 ssarkar2 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

run-test Run CI for PRs from external contributors synapse1.16

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Yantom1 @regisss @HuggingFaceDocBuilderDev @ssarkar2 @libinta