KVCache and Intelligent Routing for HyperPod Inference #290
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rvasahu-amazon
had a problem deploying
to
manual-approval
GitHub Actions
Error
…CLI (aws#252) * Initial commit for kv-cache and intelligent support in inference SDK * Initial commit for kv-cache & intelligent routing CLI changes * [Fix] clean up test code * Separate out kv-cache and intelligent routing spec update into v1_1 inference cli * Fix ut and black format * Fix template.py extra white lines
rvasahu-amazon
temporarily deployed
to
manual-approval
GitHub Actions
Inactive
rvasahu-amazon
temporarily deployed
to
manual-approval
GitHub Actions
Inactive
mollyheamazon
approved these changes
Oct 28, 2025
jam-jee
approved these changes
Oct 28, 2025
… submission changes
rvasahu-amazon
temporarily deployed
to
manual-approval
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's changing and why?
The HyperPod Inference Operator helm chart is being updated to support the release of the KVCache and Intelligent Routing features.
How was this change tested?
Helm chart changes as well as operator changes are validated by HyperPod Inference integration tests.
Are unit tests added?
No.
Are integration tests added?
No.