MGMT-21336 Query fail when a model isn't provided in the request#75
Conversation
Specify the default model and provider in the LSC config This is required because following llamastack/llama-stack#2886 llama-stack discovers the models dynamically, if we don't pass a provider and model, it falls back to the first model it finds (regardless of what we have in the llama-stack config.yaml) Note that it's also listing vertexAI models (which work with the vertexAI API)! Signed-off-by: Eran Cohen <eranco@redhat.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: eranco74 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
WalkthroughA new Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Possibly related PRs
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
lightspeed-stack.template.yaml (1)
31-32: Consider quoting the model string to prevent future YAML parsing surprisesAlthough
/is valid in unquoted scalars, quoting the value is more robust (e.g., if a future model id contains:or#).- default_model: gemini/gemini/gemini-2.0-flash + default_model: "gemini/gemini/gemini-2.0-flash"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
lightspeed-stack.template.yaml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Red Hat Konflux / assisted-chat-saas-main-on-pull-request
🔇 Additional comments (1)
lightspeed-stack.template.yaml (1)
30-32: Verify key names conform to upstream Llama-Stack schemaThe newly-added
inference:block looks correct syntactically, but upstream has changed field names more than once (default_model,default_model_name,model, etc.). A mismatch will silently fall back to the first discovered model – exactly the bug this PR tries to avoid.Please double-check against the current llama-stack schema (PR 2886) that the exact keys are:
inference: default_model: <string> default_provider: <string>and not something like
default_model_idordefault_models.
|
In which situations a model is not provided in the request? |
|
/lgtm |
integration/stage/prod |
openshift-merge-bot
bot
merged commit 8adef6a
into
rh-ecosystem-edge:main
2 participants
Specify the default model and provider in the LSC config
This is required because following
llamastack/llama-stack#2886 llama-stack discovers the models dynamically, if we don't pass a provider and model, it falls back to the first model it finds (regardless of what we have in the llama-stack config.yaml)
Note that it's also listing vertexAI models (which work with the vertexAI API)!
Summary by CodeRabbit