ci: refresh shader-validation (1.6.1) + runbook#75
Conversation
|
Warning Review limit reached
More reviews will be available in 15 minutes and 1 second. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR introduces a comprehensive runbook for validating release builds in-game using the devbench MCP test bench. The documentation covers prerequisites, build and deployment steps, in-game shader compilation and validation workflows, CI configuration refresh procedures, multi-edition coverage for SE and VR, and future automation improvements. ChangesRelease Validation Runbook
Possibly Related PRs
Poem
🎯 2 (Simple) | ⏱️ ~8 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
No actionable suggestions for changed features. |
There was a problem hiding this comment.
Pull request overview
Updates the CI shader-validation configuration to match the 1.6.1 permutation set (SE + VR) and adds a developer runbook for doing in-game release validation via devbench before publishing.
Changes:
- Regenerated
.github/configs/shader-validation.yamlandshader-validation-vr.yamlfrom fresh 1.6.1 in-game logs to keep the CI variant matrix in sync. - Added
docs/development/release-validation.mddocumenting an in-game validation flow (shader-cache gate, feature load, loadLast, log scan) plus the config refresh procedure.
Reviewed changes
Copilot reviewed 1 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
.github/configs/shader-validation.yaml |
Refreshed SE shader validation matrix/defines derived from 1.6.1 logs for CI compilation coverage. |
.github/configs/shader-validation-vr.yaml |
Refreshed VR shader validation matrix/defines derived from 1.6.1 logs for CI compilation coverage. |
docs/development/release-validation.md |
New runbook for in-game validation via devbench and for regenerating shader-validation configs when permutations change. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/development/release-validation.md (1)
1-1: ⚡ Quick winSuggest a conventional PR title within the 50-char limit.
Current title is descriptive, but over the stated limit. Suggested title:
chore(validation): refresh 1.6.1 configs + runbookAs per coding guidelines, "Conventional Commit Titles ... Length: 50 characters limit for title".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/development/release-validation.md` at line 1, PR title exceeds the 50-character conventional commit limit; update the PR title to follow the Conventional Commits format and keep it ≤50 chars — replace the current long title with something like "chore(validation): refresh 1.6.1 configs + runbook" (or an equivalent concise conventional title) so the PR title matches the required convention.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/development/release-validation.md`:
- Around line 48-50: The curl check targets the generic "inspect" tool but
should call the CS bridge's namespaced tool "openshaders.inspect" for kind=state
and kind=shadercache; update the polling commands that use the curl POST to hit
/api/tool/openshaders.inspect (keeping the same JSON body with "kind":"state" or
"kind":"shadercache") so the response fields ("plugin", "vr", "failedTasks",
"currentFailedCount") come from the correct tool; apply the same change to the
other occurrences referenced (lines ~56-60) so both checks use
openshaders.inspect consistently.
---
Nitpick comments:
In `@docs/development/release-validation.md`:
- Line 1: PR title exceeds the 50-character conventional commit limit; update
the PR title to follow the Conventional Commits format and keep it ≤50 chars —
replace the current long title with something like "chore(validation): refresh
1.6.1 configs + runbook" (or an equivalent concise conventional title) so the PR
title matches the required convention.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 4a62dd8a-b4c7-4fc0-967e-04cdda7be157
📒 Files selected for processing (3)
.github/configs/shader-validation-vr.yaml.github/configs/shader-validation.yamldocs/development/release-validation.md
- Drop the dead `project-worktree-build-maxpath` reference (Copilot); the inline MAX_PATH explanation is self-contained. - Add a note distinguishing devbench's `inspect` (state incl. playerLoaded, used for load polling) from `openshaders.inspect` (CS state/shadercache, used for the gate). The runbook's calls were already correct; this preempts the confusion CodeRabbit flagged (its suggested swap to openshaders.inspect for polling would drop playerLoaded and hang). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alandtse
changed the title
…book
Regenerate .github/configs/shader-validation{,-vr}.yaml from fresh
Debug-level CommunityShaders.log captures on the deployed 1.6.1 build (SE
and VR) via generate-shader-configs.ps1. Picks up permutation changes from
the 1.6.1 shader fixes (Lighting.hlsl, ISWaterBlend.hlsl). Both regenerated
with 0 errors (SE 24 shaders/3324 variants, VR 25/3617). Large diff is a
full mechanical regen (variant reordering + additions), not hand-edited —
the shader-validation CI job validates it by compiling the matrix.
Add docs/development/release-validation.md: runbook for in-game release
validation via the devbench MCP (shader-cache gate, feature load, in-game
load, log scan) + the config-refresh procedure, capturing the steps used
to validate 1.6.1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Drop the dead `project-worktree-build-maxpath` reference (Copilot); the inline MAX_PATH explanation is self-contained. - Add a note distinguishing devbench's `inspect` (state incl. playerLoaded, used for load polling) from `openshaders.inspect` (CS state/shadercache, used for the gate). The runbook's calls were already correct; this preempts the confusion CodeRabbit flagged (its suggested swap to openshaders.inspect for polling would drop playerLoaded and hang). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alandtse
force-pushed
the
ci/release-validation
branch
from
d7465d7 to
9dd5e8d
Compare
|
✅ A pre-release build is available for this PR: |
Regenerate both configs with the deterministic-sort hlslkit (alandtse/hlslkit#17, now on main): shaders sorted by file, entries by (entry, defines), file_common_defines keys sorted. Same variant set as before (SE 24/3324, VR 25/3617) — only ordering changes. Establishes the sorted baseline on dev so future shader-validation refreshes diff to only real added/removed variants, not full reorder churn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
What
.github/configs/shader-validation{,-vr}.yamlfrom fresh Debug-levelCommunityShaders.logcaptures on the deployed 1.6.1 build (SE + VR), viagenerate-shader-configs.ps1. Picks up permutation changes from the 1.6.1 shader fixes (Lighting.hlsl,ISWaterBlend.hlsl). Both regenerated 0 errors (SE 24 shaders / 3324 variants, VR 25 / 3617).docs/development/release-validation.md— runbook for in-game release validation via the devbench MCP (shader-cache gate, feature load, in-game load, log scan) + the config-refresh procedure.On the large diff
The config diff is a full mechanical regen (variant reordering + 1.6.1 additions; files grew SE +2.3k / VR +6k lines), not hand-edited. Don't review line-by-line — the shader-validation CI job validates it by compiling every variant in the matrix.
Validation that produced this
Built + deployed 1.6.1 (PR #74 content) to SE + VR; via devbench MCP both editions passed: 0 failed shaders, all features loaded (37 VR / 36 SE),
loadLast→ WhiterunWorld with no CTD, and 0[E]log errors traceable to 1.6.1 (only pre-existing Streamline/terrain/WeatherEditor notices).🤖 Generated with Claude Code
Summary by CodeRabbit