-
Notifications
You must be signed in to change notification settings - Fork 22
MGMT-21352: Update system prompt regarding static networking support #173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,13 @@ | |
| eval_types: [response_eval:accuracy] | ||
| expected_response: "The operators that can be installed onto clusters are OpenShift AI and OpenShift Virtualization." | ||
|
|
||
| - conversation_group: static_networking_support_conv | ||
| conversation: | ||
| - eval_id: static_networking_support | ||
| eval_query: Create a cluster with static networking | ||
| eval_types: [response_eval:sub-string] | ||
| expected_keywords: ["I do not support creating clusters with static networking", "Please use the assisted-installer web-based wizard"] | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does this work? Does it require that one of the entire quoted strings is present? That seems fragile.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It does, and it's not fragile because it's instructed to reply with these strings in the system prompt
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. still it might be better to use accuracy test here, it's more flexible. sub-string eval looks for exact matches and is not really forgiving.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TBH, I started with an accuracy test, and switched to sub-string because:
|
||
|
|
||
| - conversation_group: sno_requirements_conv | ||
| conversation: | ||
| - eval_id: sno_requirements | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Hardcoding external DNS breaks in-cluster name resolution.
Setting
dnsPolicy: "None"with Google DNS disables cluster DNS, so Pod code cannot resolve in-cluster names (Services, pods). This can break calls likehttp://lightspeed-stack:8090/and other cluster lookups, and may violate restricted-network policies.Prefer default cluster DNS, or make external resolvers optional/parametrized. Minimal safe fix:
If you truly need public resolvers, consider parameterizing them and appending via
dnsConfig.optionswhile keeping cluster DNS (and document the trade-offs).📝 Committable suggestion
🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a good point. Will this break dns resolution to the mcp server?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same thing for the UI which has:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't seem to break anything:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a good point though. I asked Gemini how it works: