Initial QAIRT model versioning - #557
Open
Kyle Romero (qti-kromero) wants to merge 10 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces initial recipe-level versioning metadata to several QAIRT Olive configuration JSONs, aiming to record a recipe version and the toolchain versions used to validate each recipe.
Changes:
- Added a new top-level
recipe_metadatablock (withversionandvalidated_with) to multiple QAIRT recipe configs. - Captured QAIRT SDK,
qairt-dev, and Python versions in each config to support reproducibility/traceability.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| microsoft-Phi-4-reasoning/QAIRT/htp_sc8480xp.json | Adds recipe_metadata for Phi-4 Reasoning on SC8480XP. |
| microsoft-Phi-4-reasoning/QAIRT/htp_sc8380xp.json | Adds recipe_metadata for Phi-4 Reasoning on SC8380XP. |
| microsoft-Phi-4-mini-instruct/QAIRT/htp_sc8380xp.json | Adds recipe_metadata for Phi-4 Mini Instruct on SC8380XP. |
| meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8480xp.json | Adds recipe_metadata for Llama 3.1 8B Instruct on SC8480XP. |
| meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8380xp.json | Adds recipe_metadata for Llama 3.1 8B Instruct on SC8380XP. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Kyle Romero (qti-kromero)
force-pushed
the
dev/qti-kromero/olive-recipe-versioning
branch
from
July 21, 2026 22:58
628e834 to
a4389e4
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (5)
microsoft-Phi-4-reasoning/QAIRT/htp_sc8480xp.json:12
- The new
recipe_metadata.validated_withvalues don’t match the validated host configuration documented inmicrosoft-Phi-4-reasoning/QAIRT/README.md(Python 3.10.12, QAIRT 2.45.40). This inconsistency can confuse users about which environment versions were actually validated; either align the JSON metadata to the README, or update the README accordingly.
"validated_with": {
"qairt_sdk": "2.45.0",
"qairt_dev": "0.5.0",
"python": "3.10"
}
microsoft-Phi-4-reasoning/QAIRT/htp_sc8380xp.json:12
- The new
recipe_metadata.validated_withvalues don’t match the validated host configuration documented inmicrosoft-Phi-4-reasoning/QAIRT/README.md(Python 3.10.12, QAIRT 2.45.40). This inconsistency can confuse users about which environment versions were actually validated; either align the JSON metadata to the README, or update the README accordingly.
"validated_with": {
"qairt_sdk": "2.45.0",
"qairt_dev": "0.5.0",
"python": "3.10"
}
microsoft-Phi-4-mini-instruct/QAIRT/htp_sc8380xp.json:13
- The new
recipe_metadata.validated_withvalues don’t match the validated host configuration documented inmicrosoft-Phi-4-mini-instruct/QAIRT/README.md(Python 3.10.12, QAIRT 2.45.40). Keeping these in sync makes it clear which versions were validated for this recipe.
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8480xp.json:12
- The new
recipe_metadata.validated_withvalues don’t match the validated host configuration documented inmeta-llama-Llama-3.1-8B-Instruct/QAIRT/README.md(Python 3.10.12, QAIRT 2.45.40). Please keep the JSON metadata and README consistent so users know which versions were validated.
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8380xp.json:9
- The new
recipe_metadata.validated_withvalues don’t match the validated host configuration documented inmeta-llama-Llama-3.1-8B-Instruct/QAIRT/README.md(Python 3.10.12, QAIRT 2.45.40). Please keep the JSON metadata and README consistent so users know which versions were validated.
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (5)
microsoft-Phi-4-reasoning/QAIRT/htp_sc8480xp.json:14
- The new
validated_withmetadata claimsqairt_sdk2.45.0 and Python 3.10, but the recipe README documents the validated host configuration as QAIRT 2.45.40 and Python 3.10.12 (microsoft-Phi-4-reasoning/QAIRT/README.md:17-19). Please confirm which versions were actually validated and align the JSON metadata (or update the README if it’s outdated).
"validated_with": {
"qairt_sdk": "2.45.0",
"qairt_dev": "0.5.0",
"python": "3.10"
}
microsoft-Phi-4-reasoning/QAIRT/htp_sc8380xp.json:14
- The new
validated_withmetadata claimsqairt_sdk2.45.0 and Python 3.10, but the recipe README documents the validated host configuration as QAIRT 2.45.40 and Python 3.10.12 (microsoft-Phi-4-reasoning/QAIRT/README.md:17-19). Please confirm which versions were actually validated and align the JSON metadata (or update the README if it’s outdated).
"validated_with": {
"qairt_sdk": "2.45.0",
"qairt_dev": "0.5.0",
"python": "3.10"
}
microsoft-Phi-4-mini-instruct/QAIRT/htp_sc8380xp.json:15
- The new
validated_withmetadata saysqairt_sdk2.48.0 and Python 3.10, but the recipe README lists the validated host configuration as QAIRT 2.45.40 and Python 3.10.12 (microsoft-Phi-4-mini-instruct/QAIRT/README.md:15-18). Please confirm which versions were actually validated and align the JSON metadata (or update the README if it’s outdated).
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8480xp.json:14
- The new
validated_withmetadata saysqairt_sdk2.48.0 and Python 3.10, but the recipe README lists the validated host configuration as QAIRT 2.45.40 and Python 3.10.12 (meta-llama-Llama-3.1-8B-Instruct/QAIRT/README.md:17-20). Please confirm which versions were actually validated and align the JSON metadata (or update the README if it’s outdated).
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
meta-llama-Llama-3.1-8B-Instruct/QAIRT/htp_sc8380xp.json:11
- The new
validated_withmetadata saysqairt_sdk2.48.0 and Python 3.10, but the recipe README lists the validated host configuration as QAIRT 2.45.40 and Python 3.10.12 (meta-llama-Llama-3.1-8B-Instruct/QAIRT/README.md:17-20). Please confirm which versions were actually validated and align the JSON metadata (or update the README if it’s outdated).
"validated_with": {
"qairt_sdk": "2.48.0",
"qairt_dev": "0.8.1",
"python": "3.10"
}
…figs recipe_metadata (version, validated_with) is now stored at the top level of each QAIRT backend's info.yml rather than inside the vendor.qairt block of individual RunConfig JSONs. This keeps recipe configs compatible with all Olive versions — the vendor field caused hard validation failures on older builds that use extra="forbid" on EngineConfig. Per-config versioning was considered and rejected: Olive discards the config filename immediately after parsing, leaving no stable in-process hook to identify which file was run.
Comment on lines
8
to
10
| "type": "QairtPreparation", | ||
| "script_path": "llama3_1_instruct_script.py" | ||
| "script_path": "llama3_1_instruct_script.py" | ||
| }, |
Correct output_dir to phi4-mini-instruct-sc8380xp and update validated_with to reflect the actual validated environment (qairt_sdk 2.45.40, python 3.10.12).
Comment on lines
+1
to
+8
| keywords: | ||
| qairt | ||
| qnn | ||
| version: "1.0.0" | ||
| validated_with: | ||
| qairt_sdk: "2.45.0" | ||
| qairt_dev: "0.5.0" | ||
| python: "3.10" |
Comment on lines
+1
to
+4
| keywords: | ||
| qairt | ||
| qnn | ||
| version: "1.0.0" |
Comment on lines
+1
to
+4
| keywords: | ||
| qairt | ||
| qnn | ||
| version: "1.0.0" |
| }, | ||
| "log_severity_level": 1, | ||
| "output_dir": "models/phi4-mini-instruct-hamoa", | ||
| "output_dir": "models/phi4-mini-instruct-sc8380xp", |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
microsoft-Phi-4-mini-instruct/QAIRT/htp_sc8380xp.json:51
output_dirnaming here deviates from the other QAIRT HTP recipes in this repo, which consistently include-qairt-htp-<soc>(e.g.,models/phi4_reasoning-qairt-htp-sc8380xpandmodels/llama3_1-8b-instruct-qairt-htp-sc8380xp). This inconsistency can make automation and artifact discovery harder and increases risk of directory collisions.
"log_severity_level": 1,
"output_dir": "models/phi4-mini-instruct-sc8380xp",
"cache_dir": "cache",
Comment on lines
+5
to
+9
| validated_with: | ||
| qairt_sdk: "2.45.40" | ||
| qairt_dev: "0.8.1" | ||
| python: "3.10" | ||
| recipes: |
Kyle Romero (qti-kromero)
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.