Skip to content

Avoid falcon perf drop from PR#607 when BS=1#620

Merged
regisss merged 4 commits into
mainfrom
dev/schoi/falcon_larger_bs
Jan 3, 2024
Merged

Avoid falcon perf drop from PR#607 when BS=1#620
regisss merged 4 commits into
mainfrom
dev/schoi/falcon_larger_bs

Conversation

@schoi-habana
Copy link
Copy Markdown
Collaborator

@schoi-habana schoi-habana commented Jan 2, 2024

#607 fixed the graph compilation error for BS>1 but it degrades the perf when BS=1.
So an additional condition was added to apply the change in PR#607 only when BS>1

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@schoi-habana schoi-habana marked this pull request as ready for review January 3, 2024 00:00
@schoi-habana schoi-habana requested review from a user and regisss January 3, 2024 00:00
Copy link
Copy Markdown
Collaborator

@mandy-li mandy-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@regisss regisss merged commit b0421c1 into main Jan 3, 2024
@regisss regisss deleted the dev/schoi/falcon_larger_bs branch January 3, 2024 09:55
jychen21 pushed a commit to jychen21/optimum-habana that referenced this pull request Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants