Skip to content

Conversation

@eranco74
Copy link
Collaborator

@eranco74 eranco74 commented Sep 15, 2025

This commit refines the system prompt to align the AI's behavior with the desired brand voice, moving from a purely functional assistant to a more collaborative and user-centric partner. The changes are based on benchmark examples that score well for being conversational, empowering, and context-aware.

Key Improvements:

  • Enrich Persona: The core persona is updated to be more collaborative, professional, and empathetic, especially during troubleshooting scenarios.
  • Improve UX for Lists: The assistant now summarizes long lists (e.g., events) instead of displaying them in full, preventing user overload.
  • Increase Robustness: New principles were added to handle ambiguous user input and to require the assistant to confirm the success or failure of its actions based on evidence from tool outputs.

Summary by CodeRabbit

  • Improvements
    • Enhanced AI persona for concise, peer-like collaboration with clearer clarifying questions, concrete options, and candid limits.
    • Improved list handling: short lists shown in full; long lists summarized with options to reveal, filter, or show recent/critical items.
    • Added observability: actions now require explicit validation with concrete evidence (e.g., exact outcomes or IDs) rather than generic confirmations.

@openshift-ci openshift-ci bot requested review from jhernand and omertuc September 15, 2025 18:39
@coderabbitai
Copy link

coderabbitai bot commented Sep 15, 2025

Caution

Review failed

The head commit changed during the review from 1b5cce0 to ab61fff.

Walkthrough

Updated the template.yaml system prompt: expanded assistant persona and conversational tone guidance, revised list-output display rules (full for ≤10, summarized for >10 with reveal/filter options), and added an observability/validation directive requiring evidence-based confirmations. No code or API changes.

Changes

Cohort / File(s) Change Summary
AI Assistant System Prompt
template.yaml
Expanded identity/persona guidance (concise, peer-like, ask clarifying questions, present multiple approaches); updated list display rules (show ≤10 items in full, summarize >10 with reveal/filter/recent options); added observability/validation directive to require tool-evidence and exact outcomes; retained policy blocks.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested labels

lgtm

Suggested reviewers

  • jhernand
  • maorfr
  • carbonin
  • omertuc

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The pull request title "feat(prompt): Enhance assistant persona and user experience" is directly aligned with the primary objectives of the changeset. The PR specifically enhances the system prompt to make the assistant more collaborative, empowering, and user-centric (persona improvements), while also improving list output handling and adding action validation (user experience improvements). The title is concise, clear, and uses specific language ("Enhance assistant persona and user experience") rather than vague terminology, making it sufficiently descriptive for someone scanning the repository history to understand the main change at a glance.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link

openshift-ci bot commented Sep 15, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: eranco74

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@eranco74
Copy link
Collaborator Author

/hold
Just checking

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
template.yaml (2)

434-435: Parameter interpolation bug: use ${VAR}, not ${{VAR}}.

OpenShift Templates substitute ${PARAM}. ${{…}} will not resolve, breaking replicas and ports and potentially failing probes and Service wiring.

-    replicas: ${{REPLICAS_COUNT}}
+    replicas: ${REPLICAS_COUNT}
...
-            containerPort: ${{SERVICE_PORT}}
+            containerPort: ${SERVICE_PORT}
...
-              port: ${{SERVICE_PORT}}
+              port: ${SERVICE_PORT}
...
-              port: ${{SERVICE_PORT}}
+              port: ${SERVICE_PORT}
...
-      port: ${{SERVICE_PORT}}
-      targetPort: ${{SERVICE_PORT}}
+      port: ${SERVICE_PORT}
+      targetPort: ${SERVICE_PORT}

Also applies to: 450-452, 514-515, 521-522, 602-603


599-606: Headless Service prevents Route from working.

Routes require a Service with a ClusterIP. clusterIP: None makes it headless and typically unroutable.

-  spec:
-    clusterIP: None
+  spec:
🧹 Nitpick comments (4)
template.yaml (4)

238-241: Add a privacy/safety caveat when displaying lists.

Explicitly note that lists must not expose sensitive links (e.g., ISO/kubeconfig URLs) even for short lists.

       **When a tool provides a list of items, your primary response is to present the information clearly.**
       * **For short lists (e.g., 10 items or fewer),** present the complete list directly to the user.
-      * **For long lists (over 10 items),** present a summary (e.g., "I found 25 events, with 3 critical errors.") and then offer to display the full list, filter it, or show the most recent/critical items.
+      * **For long lists (over 10 items),** present a summary (e.g., "I found 25 events, with 3 critical errors.") and then offer to display the full list, filter it, or show the most recent/critical items.
+      * If a list contains sensitive values (for example, Discovery ISO or kubeconfig URLs), summarize items without revealing the URLs.

274-279: Qualify hardware requirements as version‑dependent minimums and avoid over‑precision.

These numbers vary by OpenShift release and operator set; hardcoding can mislead.

-              * For a multi-node cluster:
+              * Minimums (vary by OpenShift version; operator choices may increase these):
                 * Control plane (master) nodes require: 4 CPU cores, 16 GB RAM, 100 GB storage
                 * Compute (worker) nodes require: 2 CPU cores, 8 GB RAM, 100 GB storage
-              * For a single node cluster (SNO): 8 CPU cores, 16 GB RAM, 100 GB storage
+              * For a single node cluster (SNO): 8 CPU cores, 16–32 GB RAM (recommended 32 GB), 100 GB storage

300-301: Troubleshooting flow tweak (optional).

Recommend “events first, then logs” to minimize heavy downloads, and prompt for cluster/host IDs if ambiguous.


131-136: Typo in parameter name: LIGHTSSPEED → LIGHTSPEED.

Spelling inconsistency invites future misconfiguration. Consider renaming the parameter and its reference (breaking change) or add a migration note.

-- name: LIGHTSSPEED_STACK_POSTGRES_SSL_MODE
+- name: LIGHTSPEED_STACK_POSTGRES_SSL_MODE
   value: "verify-full"
   description: "SSL mode for the PostgreSQL database connection used by lightspeed-stack"
...
-          ssl_mode: ${LIGHTSSPEED_STACK_POSTGRES_SSL_MODE}
+          ssl_mode: ${LIGHTSPEED_STACK_POSTGRES_SSL_MODE}

Also applies to: 198-199

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3cdf477 and 19d6ded.

📒 Files selected for processing (1)
  • template.yaml (6 hunks)
🔇 Additional comments (4)
template.yaml (4)

209-209: Persona adjustments LGTM.

Collaborative, empathetic tone guidance is clear and aligned with the PR goal.


296-296: Kubeconfig/kubeadmin URL non-disclosure LGTM.


310-311: Ambiguity-handling and evidence-based confirmation LGTM.


256-257: Narrow refusal of “static networking” — refuse only when tooling truly lacks support.

  • Do not refuse static networking wholesale. Check whether available tooling in this repo (e.g., nmstate/MCP/assisted‑installer support) can satisfy node-level network parameters; if tooling cannot, explicitly state this is a tooling limitation and direct the user to the assisted‑installer web wizard.
  • Only trigger refusal for explicit node-level overrides (static IPs, MAC addresses, gateways, DNS, or other node-specific network settings).

File: template.yaml lines 256-257

-          * **If an unsupported configuration is detected, you must refuse the entire request.** Do not proceed with cluster creation. Your response must state that you don't support the requested static networking configuration and then instruct the user to use the assisted-installer web-based wizard instead.
+          * **If the requested configuration is not supported by available tools,** explain the limitation and guide the user to the assisted‑installer web wizard. Otherwise proceed.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
template.yaml (3)

206-206: Resolve tone guidance conflict; prefer “plain language, defined terms” over “avoid jargon.”

Line 206 partly contradicts Line 205 (“use technical language, spell out abbreviations”). Tighten to “avoid unexplained jargon; define terms on first use” to remove ambiguity.

Apply:

-      Be a helpful colleague. Get straight to the point but engage the user like a peer. Ask clarifying questions if you need more information to give a precise answer. For example, "What version are you using?" Give concrete advice, not vague platitudes. Use specific commands, examples, and numbers whenever possible. Present options. If there are multiple valid approaches, lay them out clearly. Don't overpromise. Be honest about limitations and what you can't do. Be conversational. Use contractions and avoid jargon. Empower the user. Position Red Hat's technology as an enabler for their success, not a replacement for their expertise.
+      Be a helpful colleague. Get straight to the point but engage the user like a peer. Ask clarifying questions if you need more information to give a precise answer (e.g., “What version are you using?”). Give concrete advice with specific commands, examples, and numbers whenever possible. Present options when multiple approaches are valid. Don’t overpromise—be honest about limitations. Use plain language and avoid unexplained jargon; define domain terms and abbreviations on first use. Empower the user—position Red Hat’s technology as an enabler for their success, not a replacement for their expertise.

236-238: Align list‑display rule wording with “don’t mention tools” policy; add safe default cap.

Avoid “tool” wording in the instruction itself and set a default top‑N to prevent flooding.

Apply:

-      **When a tool provides a list of items, your primary response is to present the information clearly.**
+      **When you need to present a list returned from internal operations, present the information clearly without mentioning internal tools.**
       * **For short lists (e.g., 10 items or fewer),** present the complete list directly to the user.
-      * **For long lists (over 10 items),** present a summary (e.g., "I found 25 events, with 3 critical errors.") and then offer to display the full list, filter it, or show the most recent/critical items.
+      * **For long lists (over 10 items),** present a one‑line summary (e.g., “I found 25 events; 3 critical.”), then show the top 10 most recent or critical items by default. Offer to display the full list, apply filters, or change the sort.

311-313: Great additions; also instruct to include evidence timestamps.

This strengthens the “confirm with evidence” rule and improves auditability.

Apply:

-      * **Confirm with Evidence: After executing an action (like creating a cluster or assigning a host role), always use the output from the tool to confirm its success or failure. Do not simply assume the action worked. For example, say "I have created the cluster 'my-cluster' with ID: xyz-123," not just "I have created the cluster."**
+      * **Confirm with Evidence: After executing an action (like creating a cluster or assigning a host role), always use the output to confirm success or failure. Do not assume it worked. For example: “I created the cluster ‘my-cluster’ (ID: xyz-123).” Include relevant timestamps or status versions where applicable.**
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 19d6ded and d82f42e.

📒 Files selected for processing (1)
  • template.yaml (4 hunks)
🔇 Additional comments (1)
template.yaml (1)

271-275: Do not raise SNO minimum RAM — match Red Hat's documented minimums and label values as "minimums".

Red Hat (OpenShift 4.18) lists SNO minimums as 8 vCPUs, 16 GB RAM, 120 GB storage. Update template.yaml (lines 271–275) to mark multi-node and SNO values as "minimums" and change SNO storage to 120 GB. If you want to advise more RAM for heavier workloads, add a separate "recommended" note (32 GB as recommended, not minimum).

Likely an incorrect or invalid review comment.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
template.yaml (1)

201-340: Line 207 indentation is still incorrect—requires 1 additional space.

Line 207 currently has 5 leading spaces but should have 6 to match the rest of the literal block. Change:

     Be honest about limitations...

to:

      Be honest about limitations...

This will resolve the yamllint syntax error: expected <block end>, but found '<scalar>'

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d82f42e and 35ffc97.

📒 Files selected for processing (1)
  • template.yaml (3 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
template.yaml

[error] 207-207: syntax error: expected , but found ''

(syntax)

🔇 Additional comments (3)
template.yaml (3)

206-207: Persona guidance enhancements are clear and well-structured.

Once the indentation error is fixed, the collaborative peer-like framing (concrete advice, asking clarifying questions, avoiding overpromising, and emphasizing honesty about limitations) strengthens the assistant's positioning. This aligns well with the PR objectives.


235-237: List output guidance provides clear UX improvement.

The distinction between short lists (≤10 items, display in full) and long lists (>10 items, summarize with options) is practical and helps prevent information overload. This is a good addition to the prompt.


339-339: Evidence-based confirmation principle strengthens observability.

Requiring the assistant to confirm actions using tool outputs (e.g., "I have created the cluster 'my-cluster' with ID: xyz-123" rather than generic success messages) improves trustworthiness and debuggability. Well-reasoned addition.

@eranco74
Copy link
Collaborator Author

/retest

This commit refines the system prompt to align the AI's behavior with the desired brand voice, moving from a purely functional assistant to a more collaborative and user-centric partner.
The changes are based on benchmark examples that score well for being conversational, empowering, and context-aware.

Key Improvements:
- **Enrich Persona:** The core persona is updated to be more collaborative, professional, and empathetic, especially during troubleshooting scenarios.
- **Improve UX for Lists:** The assistant now summarizes long lists (e.g., events) instead of displaying them in full, preventing user overload.
- **Increase Robustness:** Confirm the success or failure of an actions based on evidence from tool outputs.
@eranco74
Copy link
Collaborator Author

Keep failing with:

Error from server: Get "https://10.0.44.115:10250/containerLogs/ci-op-f8db18rz/assisted-chat-eval-test-58l1ovtl-68psk/assisted-chat-eval-test": tls: failed to verify certificate: x509: certificate signed by unknown authority

/retest

@eranco74
Copy link
Collaborator Author

/retest

1 similar comment
@eranco74
Copy link
Collaborator Author

/retest

@eranco74
Copy link
Collaborator Author

2/3 test failed due to: Error message: Unexpected error in streaming query: [Errno 104] Connection reset by peer

capabilities_scope_conv probably needs some adjustments in the expected response
/retest

@eranco74 eranco74 force-pushed the system_prompt branch 2 times, most recently from 1b5cce0 to ab61fff Compare November 18, 2025 14:53
@openshift-ci
Copy link

openshift-ci bot commented Nov 18, 2025

@eranco74: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/eval-test ab61fff link false /test eval-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants