Skip to content

Updted the template to use the LLAMA_STACK_2_0_FLASH_MODEL as the default model #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-bot merged 1 commit into from
Jul 30, 2025
Merged

Updted the template to use the LLAMA_STACK_2_0_FLASH_MODEL as the default model #77

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,15 @@ parameters:
- name: LLAMA_STACK_INFERENCE_PROVIDER_TYPE
value: "remote::gemini"
description: "Type specification for the inference provider (remote::gemini for Google Gemini)"
- name: LLAMA_STACK_DEFAULT_MODEL
- name: LLAMA_STACK_2_5_PRO_MODEL
value: "gemini/gemini-2.5-pro"
description: "Default model to use for inference requests"
- name: LLAMA_STACK_FLASH_MODEL
- name: LLAMA_STACK_2_5_FLASH_MODEL
value: "gemini/gemini-2.5-flash"
description: "Fast model to use for quick inference requests"
- name: LLAMA_STACK_2_0_FLASH_MODEL
value: "gemini/gemini-2.0-flash"
description: "Fast model to use for quick inference requests"
- name: LLAMA_STACK_SERVER_PORT
value: "8321"
description: "Port number for the embedded Llama Stack server"
Expand Down Expand Up @@ -140,8 +143,8 @@ objects:
system_prompt_path: "/app-root/system_prompt"
disable_query_system_prompt: true
inference:
default_model: gemini/gemini/gemini-2.0-flash
default_provider: gemini
default_model: "${LLAMA_STACK_INFERENCE_PROVIDER}/${LLAMA_STACK_2_0_FLASH_MODEL}"
default_provider: ${LLAMA_STACK_INFERENCE_PROVIDER}
Comment on lines +146 to +147
Copy link

@coderabbitai coderabbitai bot Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

default_model builds an invalid path (gemini/gemini/...).

Because ${LLAMA_STACK_2_0_FLASH_MODEL} already starts with
gemini/, prepending ${LLAMA_STACK_INFERENCE_PROVIDER}/
duplicates the provider segment and will break routing to the model.

-        default_model: "${LLAMA_STACK_INFERENCE_PROVIDER}/${LLAMA_STACK_2_0_FLASH_MODEL}"
+        # Param already includes provider – no need to prepend it
+        default_model: "${LLAMA_STACK_2_0_FLASH_MODEL}"

Apply the same rule wherever the “MODEL” parameters are concatenated
with the provider.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
default_model: "${LLAMA_STACK_INFERENCE_PROVIDER}/${LLAMA_STACK_2_0_FLASH_MODEL}"
default_provider: ${LLAMA_STACK_INFERENCE_PROVIDER}
# Param already includes provider – no need to prepend it
default_model: "${LLAMA_STACK_2_0_FLASH_MODEL}"
default_provider: ${LLAMA_STACK_INFERENCE_PROVIDER}
🤖 Prompt for AI Agents 
In template.yaml around lines 146 to 147, the default_model value concatenates
the provider and model variables, causing duplication of the provider segment in
the path. To fix this, remove the provider prefix from default_model and use
only the model variable since it already includes the provider segment. Apply
this correction consistently wherever model parameters are concatenated with the
provider to avoid invalid paths.

Copy link
Collaborator Author

@eranco74 eranco74 Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We know it's strange, we have a task for that
https://issues.redhat.com/browse/MGMT-21341

system_prompt: |
You are OpenShift Lightspeed Intelligent Assistant - an intelligent virtual assistant and expert on all things related to OpenShift installation, configuration, and troubleshooting, specifically with the Assisted Installer.

Expand Down Expand Up @@ -297,14 +300,19 @@ objects:
password: ${env.LLAMA_STACK_POSTGRES_PASSWORD}
models:
- metadata: {}
model_id: ${LLAMA_STACK_DEFAULT_MODEL}
model_id: ${LLAMA_STACK_2_0_FLASH_MODEL}
provider_id: ${LLAMA_STACK_INFERENCE_PROVIDER}
provider_model_id: ${LLAMA_STACK_2_0_FLASH_MODEL}
model_type: llm
- metadata: {}
model_id: ${LLAMA_STACK_2_5_PRO_MODEL}
provider_id: ${LLAMA_STACK_INFERENCE_PROVIDER}
provider_model_id: ${LLAMA_STACK_DEFAULT_MODEL}
provider_model_id: ${LLAMA_STACK_2_5_PRO_MODEL}
model_type: llm
- metadata: {}
model_id: ${LLAMA_STACK_FLASH_MODEL}
model_id: ${LLAMA_STACK_2_5_FLASH_MODEL}
provider_id: ${LLAMA_STACK_INFERENCE_PROVIDER}
provider_model_id: ${LLAMA_STACK_FLASH_MODEL}
provider_model_id: ${LLAMA_STACK_2_5_FLASH_MODEL}
model_type: llm
shields: []
vector_dbs: []
Expand Down