Skip to content

fix: update eval-address-reviews EVAL_CONFIG path for ai-helpers - #81069

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
bryan-cox:fix-eval-config-path
Jun 25, 2026
Merged

openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
bryan-cox:fix-eval-config-path

Conversation

@bryan-cox

@bryan-cox bryan-cox commented Jun 25, 2026

Copy link
Copy Markdown
Member

The eval-address-reviews job's EVAL_CONFIG points to plugins/utils/evals/eval-address-reviews.yaml but that file no longer exists on main (removed by openshift-eng/ai-helpers#576).

openshift-eng/ai-helpers#558 adds the eval config at plugins/openshift-developer/evals/eval-address-reviews.yaml. This PR updates the path to match so the eval job can run against PR #558.

The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The eval-address-reviews CI job now uses plugins/openshift-developer/evals/eval-address-reviews.yaml instead of the previous plugins/utils/evals/eval-address-reviews.yaml path.

Changes

AI helpers eval config path update

Layer / File(s) Summary
EVAL_CONFIG path update
ci-operator/config/openshift-eng/ai-helpers/openshift-eng-ai-helpers-main.yaml
eval-address-reviews now points EVAL_CONFIG at plugins/openshift-developer/evals/eval-address-reviews.yaml.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

  • openshift/release#80842: Changes the same eval-address-reviews CI wiring and EVAL_CONFIG path in the same config file.

Suggested labels

lgtm, approved, rehearsals-ack

Suggested reviewers

  • bennerv
  • ashishmax31
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: updating the eval-address-reviews EVAL_CONFIG path in ai-helpers.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Only a CI YAML env var changed; no Ginkgo test definitions or titles were added/modified, so there are no unstable test names to flag.
Test Structure And Quality ✅ Passed Only ci-operator YAML/OWNERS were touched; no Ginkgo tests or assertions were modified, so this test-structure check is not applicable.
Microshift Test Compatibility ✅ Passed PASS: This PR only updates a ci-operator EVAL_CONFIG path in YAML; no Ginkgo/e2e tests or MicroShift-relevant APIs were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR only updates a CI EVAL_CONFIG path in YAML; no new Ginkgo e2e tests or node-topology assumptions were added.
Topology-Aware Scheduling Compatibility ✅ Passed Only a CI job EVAL_CONFIG path changed; no pod scheduling, replicas, affinity, nodeSelector, or PDB settings were introduced.
Ote Binary Stdout Contract ✅ Passed PASS: PR only updates a CI job YAML EVAL_CONFIG path; no process-level code, stdout writes, or suite setup changes are present.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR only updates a ci-operator YAML EVAL_CONFIG path; it adds no Ginkgo tests or network-dependent e2e code, so the IPv6/disconnected check is not applicable.
No-Weak-Crypto ✅ Passed Only a CI YAML EVAL_CONFIG path changed; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons found.
Container-Privileges ✅ Passed PASS: The patch only updates an EVAL_CONFIG path in a CI job; no privileged fields (privileged, hostPID, hostNetwork, hostIPC, allowPrivilegeEscalation, SYS_ADMIN) appear in the changed manifest.
No-Sensitive-Data-In-Logs ✅ Passed The change only updates an EVAL_CONFIG path in a Prow job YAML; no logging or sensitive-data handling was added.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from enxebre and stbenjam June 25, 2026 11:17
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 25, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@bryan-cox: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-eng-ai-helpers-main-eval-address-reviews openshift-eng/ai-helpers presubmit Ci-operator config changed

Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@enxebre

enxebre commented Jun 25, 2026

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox, enxebre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@bryan-cox

Copy link
Copy Markdown
Member Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@bryan-cox: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 25, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 68960f4 into openshift:main Jun 25, 2026
14 checks passed
kasturinarra pushed a commit to kasturinarra/release that referenced this pull request Jun 26, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
krisnababu pushed a commit to oharan2/release that referenced this pull request Jul 3, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
andrej1991 pushed a commit to andrej1991/release that referenced this pull request Jul 6, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Jul 8, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
SachinNinganure pushed a commit to SachinNinganure/release that referenced this pull request Jul 20, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
amogh-redhat pushed a commit to amogh-redhat/release that referenced this pull request Aug 5, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Sep 5, 2026
The address-reviews eval config moved from plugins/utils/evals/ to
plugins/openshift-developer/evals/ when the skill was relocated to
the openshift-developer bundle plugin.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants