Enable long CL for Phi 3 recipes - #574
Conversation
|
ssruti02 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
Pull request overview
This PR aims to enable long context-length (long CL) support for the QNN optimization workflows of two Phi-3 family recipes by switching the RemoveRopeMultiCache graph surgery to use a larger cache.
Changes:
- Adds
"use_large_cache": trueto theRemoveRopeMultiCachesurgery in the Phi-3.5 Mini Instruct QNN config. - Adds
"use_large_cache": trueto theRemoveRopeMultiCachesurgery in the Phi-3 Mini 128k Instruct QNN config.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| microsoft-Phi-3.5-mini-instruct/QNN/x_elite_config.json | Enables large RoPE cache in the graph surgery configuration (intended to support long CL). |
| microsoft-Phi-3-mini-128k-instruct/QNN/config.json | Enables large RoPE cache in the graph surgery configuration (intended to support long CL). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Enables long CL support for the following recipes:
Updates the corresponding QNN configuration files for both models.