MGMT-21732: Reenforce use of tool calls

[keitwb]

Sometimes the model just punts on the tool calls and asks the user to do them for some reason. This is already forbidden in the prompt so this is reenforcing it from the opposite angle of encouraging it to do the tool calls itself.

Summary by CodeRabbit

• New Features

  • Assistant can now proactively use tools to gather missing details when needed.

• Documentation

  • Updated guidance to reflect streamlined interaction: removed step-by-step success confirmations.
  • Simplified failure-handling guidance to reduce verbosity and focus on concise responses.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: keitwb
Once this PR has been reviewed and has the lgtm label, please assign maorfr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Walkthrough

Edits to template.yaml adjust prompt guidance: allow learning missing details via tool calls, and remove directives about post-step success confirmation and explicit failure-handling instructions.

Changes

Cohort / File(s)	Change summary
Prompt template updates template.yaml	Modified “General Proactive Principles”: added allowance for learning missing info via tool calls; removed guidance on post-step success confirmation and explicit failure-handling instructions.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• Improve cluster id/name handling #188 — Also updates template.yaml to permit obtaining missing info via tool calls in a cluster name-to-ID context.
• MGMT-21392: avoid referencing function names #123 — Modifies template.yaml guidance around tool usage, adding restrictions on exposing tool names.

Suggested labels

ok-to-test

Suggested reviewers

• eranco74
• omertuc
• carbonin
• zszabo-rh

Pre-merge checks (3 passed)

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "MGMT-21732: Reenforce use of tool calls" succinctly and accurately reflects the PR’s primary objective of strengthening encouragement for the model to invoke tool calls, which matches the template.yaml edits and the PR description. It is concise, focused, and includes the ticket reference for traceability.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci-robot]

@keitwb: This pull request references MGMT-21732 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.21.0" version, but no target version was set.

Details

In response to this:

Sometimes the model just punts on the tool calls and asks the user to do them for some reason. This is already forbidden in the prompt so this is reenforcing it from the opposite angle of encouraging it to do the tool calls itself.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@keitwb: This pull request references MGMT-21732 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.21.0" version, but no target version was set.

Details

In response to this:

Sometimes the model just punts on the tool calls and asks the user to do them for some reason. This is already forbidden in the prompt so this is reenforcing it from the opposite angle of encouraging it to do the tool calls itself.

Summary by CodeRabbit

• New Features

• Assistant can now proactively use tools to gather missing details when needed.

• Documentation

• Updated guidance to reflect streamlined interaction: removed step-by-step success confirmations.

• Simplified failure-handling guidance to reduce verbosity and focus on concise responses.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

template.yaml (1)

308-309: Strengthen wording to mandate self-invoked tool calls; fix capitalization.

The added guidance is good. To fully align with the PR objective (model invokes tools itself) and for consistency, tighten the language and capitalize “OpenShift”.

-      * If a step requires specific information (e.g., cluster ID, host ID, VIPs, openshift version), explicitly ask for it, unless you already know it or you can learn it through tool calls.
+      * If a step requires specific information (e.g., cluster ID, host ID, VIPs, OpenShift version), explicitly ask for it only if it isn’t already known or retrievable via tool calls. Prefer invoking tool calls yourself rather than asking the user to perform any tooling.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e1cf7fb and 58df9d3.

📒 Files selected for processing (1)

• template.yaml (1 hunks)

🔇 Additional comments (1)

template.yaml (1)

311-313: AI summary inconsistency: success/failure guidance still present.

The AI-generated summary says the “post-step success confirmation” and “explicit failure-handling” bullets were removed, but Lines 311–313 still include them. Confirm intent: keep these bullets (recommended), or remove them to match the summary.

[oourfali]

/lgtm

[eranco74]

/retest

[openshift-ci]

@keitwb: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/eval-test	58df9d3	link	false	/test eval-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.