Universal Speculative Decoding `CandidateGenerator`#35029

Merged

gante merged 112 commits intohuggingface:mainfrom

keyboardAnt:usd

Feb 26, 2025

Commits on Dec 3, 2024

move `TestAssistedCandidateGeneratorDifferentTokenizers` into a new testing file
Nadav Timor
committed
refactor
Nadav Timor
committed
NOTHING. add space to rerun github actions tests
Nadav Timor
committed
remove it...
Nadav Timor
committed
`UniversalSpeculativeDecodingGenerator`
Nadav Timor
committed
Use `UniversalSpeculativeDecodingGenerator` when `generation_config.do_sample=True`
Nadav Timor
committed
assistant tokenizes only the target's new suffix
Nadav Timor
committed
formatting
Nadav Timor
committed
fix code

jmamou
authored and
Nadav Timor
committed
fix code

jmamou
authored and
Nadav Timor
committed
formatting
Nadav Timor
committed
add `TestGenerateWithDifferentModels`
Nadav Timor
committed
`TestGenerateWithDifferentModels` parameterize on `do_sample`
Nadav Timor
committed
`AssistantVocabMapping` & `AssistantVocabMappingCache`
Nadav Timor
committed
formatting
Nadav Timor
committed
`AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_logits`
Nadav Timor
committed
improve `_get_assistant_to_target_input_ids` & formatting
Nadav Timor
committed
renaming
Nadav Timor
committed
WIP: debugging `min_new_tokens`
Nadav Timor
committed
fix get_target_ids

jmamou
authored and
Nadav Timor
committed
`UniversalSpeculativeDecodingGenerator`
Nadav Timor
committed
assistant tokenizes only the target's new suffix
Nadav Timor
committed
formatting
Nadav Timor
committed
fix code

jmamou
authored and
Nadav Timor
committed
fix code

jmamou
authored and
Nadav Timor
committed
formatting
Nadav Timor
committed
`TestGenerateWithDifferentModels` parameterize on `do_sample`
Nadav Timor
committed
`AssistantVocabMapping` & `AssistantVocabMappingCache`
Nadav Timor
committed
formatting
Nadav Timor
committed
`AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_logits`
Nadav Timor
committed
improve `_get_assistant_to_target_input_ids` & formatting
Nadav Timor
committed
renaming
Nadav Timor
committed
WIP: debugging `min_new_tokens`
Nadav Timor
committed
fix get_target_ids

jmamou
authored and
Nadav Timor
committed
fix device issue

jmamou
authored and
Nadav Timor
committed
fix get_assistant_input_ids

jmamou
authored and
Nadav Timor
committed
add `TestAssistedCandidateGeneratorDifferentTokenizers`
Nadav Timor
committed
formatting
Nadav Timor
committed
`AssistantVocabTranslatorCache` refactor & tests
Nadav Timor
committed
revert changes in `src/transformers/generation/logits_process.py`
Nadav Timor
committed
refactor `AssistedCandidateGenerator`
Nadav Timor
committed
refactor `AssistedCandidateGeneratorDifferentTokenizers`
Nadav Timor
committed
formatting
Nadav Timor
committed
refactor `UniversalSpeculativeDecodingGenerator`
Nadav Timor
committed
fix negative value for max_new_tokens

jmamou
authored and
Nadav Timor
committed
fix generation length target + attention_mask vs. assistant + attent

jmamou
authored and
Nadav Timor
committed
fix device

jmamou
authored and
Nadav Timor
committed
fix negative max_new_tokens bug

jmamou
authored and
Nadav Timor
committed
fix UAG

jmamou
authored and
Nadav Timor
committed
minor

jmamou
authored and
Nadav Timor
committed
formatting
Nadav Timor
committed
`AssistedCandidateGeneratorDifferentTokenizers` `lookbehind`s init
Nadav Timor
committed
resolve conflict & formatting
Nadav Timor
committed
rerun CI tests
Nadav Timor
committed
remove space...
Nadav Timor
committed
remove old code
Nadav Timor
committed

Commits on Dec 4, 2024

Commits on Dec 5, 2024

Merge pull request #4 from keyboardAnt/fix_device
keyboardAnt
authored
formatting
Nadav Timor
committed

Commits on Dec 17, 2024

Fix prepare + apply (#7)
jmamou
authored

Commits on Dec 19, 2024

Commits on Dec 20, 2024

Remove unused imports and fix style using `make style` (#9)
gauravjain14
authored

Commits on Dec 21, 2024

formatting
Nadav Timor
committed
Swap gated `meta-llama/llama-3.2` with `allenai/llama` (#10)
gauravjain14
authored

Commits on Jan 6, 2025

Merge branch 'main' into usd
Nadav Timor
committed

Commits on Jan 9, 2025

Commits on Jan 12, 2025

Commits on Jan 13, 2025

typo (`target_to_assistant_input_ids`)
Nadav Timor
committed
formatting
Nadav Timor
committed

Commits on Jan 15, 2025

merge upstream/main
Nadav Timor
committed

Commits on Jan 16, 2025

Merge branch 'main' into usd
Nadav Timor
committed

Commits on Jan 27, 2025

Fix minor review comments (#16)
gauravjain14
authored

Commits on Jan 28, 2025

Fix: `token_ids.to(torch.int64)` (#18)
keyboardAnt
authored
fix dtype
Nadav Timor
committed
`assistant_input_ids.to(dtype=torch.long)`
Nadav Timor
committed
Remove unused import from test_candidate_generator.py
gauravjain14
authored

Commits on Feb 15, 2025

Merge branch 'main' of https://github.com/keyboardAnt/transformers into usd
Nadav Timor
committed
resolve pr comments (#19)
keyboardAnt
authored
formatting
Nadav Timor
committed

Commits on Feb 23, 2025

Merge branch 'main' into usd
keyboardAnt
authored

Commits on Feb 24, 2025

Merge branch 'main' into usd
jmamou
authored

Commits on Feb 25, 2025