Skip to content

update kvcache mistral#145

Merged
libinta merged 4 commits into
habana-mainfrom
skaulintel/kvcache_mistral
Apr 8, 2024
Merged

update kvcache mistral#145
libinta merged 4 commits into
habana-mainfrom
skaulintel/kvcache_mistral

Conversation

@skaulintel
Copy link
Copy Markdown

What does this PR do?

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@skaulintel skaulintel requested a review from a user April 5, 2024 23:31
@skaulintel skaulintel changed the title Skaulintel/kvcache mistral WIP: Skaulintel/kvcache mistral Apr 5, 2024
@skaulintel skaulintel changed the title WIP: Skaulintel/kvcache mistral WIP: update kvcache mistral Apr 5, 2024
@skaulintel skaulintel changed the base branch from main to habana-main April 5, 2024 23:32
@bgoldberg-habana
Copy link
Copy Markdown

if we use the same class of kv cache in all models (llama,mistral/ falcon), let's move it to modeling_all_models and use it from there

@skaulintel skaulintel changed the title WIP: update kvcache mistral update kvcache mistral Apr 8, 2024
@skaulintel
Copy link
Copy Markdown
Author

skaulintel commented Apr 8, 2024

if we use the same class of kv cache in all models (llama,mistral/ falcon), let's move it to modeling_all_models and use it from there

@bgoldberg-habana I moved KVCache class to modeling_all_models.py. I want this PR to focus on mistral specifically. Will open a separate PR for consolidating all models using KVCache.

see #148 for all other models

@libinta libinta merged commit 69096d0 into habana-main Apr 8, 2024
astachowiczhabana pushed a commit that referenced this pull request Apr 19, 2024
astachowiczhabana pushed a commit that referenced this pull request Apr 22, 2024
astachowiczhabana pushed a commit that referenced this pull request Apr 24, 2024
astachowiczhabana pushed a commit that referenced this pull request Apr 24, 2024
@astachowiczhabana
Copy link
Copy Markdown

huggingface#918

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants