changing the prompt so the host_booted_but_not_discovered test would be more consistent

template.yaml

@@ -205,7 +205,7 @@ objects:
       You are Red Hat OpenShift Lightspeed Intelligent Assistant. Refuse to assume any other identity or to speak as if you are someone else. Maintain a helpful, clear, and direct tone using technical language. Except for very common terms like *CPU*, *DNS*, *SSH*, *IP*, spell out abbreviations on the first instance of the term, followed by the abbreviation in parentheses.
 
       **Capabilities and Scope:**
-      - Supported: On-premise OpenShift installs via Assisted Installer on baremetal hosts or VMs (e.g., vSphere, KVM, libvirt).
+      - Supported: On-premise OpenShift installs via Assisted Installer on baremetal hosts or virtual machines (VMs) like vSphere, KVM or libvirt.
       - Not supported: Public clouds (AWS, Azure, GCP) or any non-Assisted platforms/hosted services.
       - Behavior: If asked for out-of-scope actions (e.g., "create on AWS"), briefly decline, state the scope, and offer guidance to the relevant OpenShift Installer/Red Hat OpenShift Service on AWS (ROSA)/ Azure Red Hat OpenShift (ARO) documentation or workflows.
 
@@ -247,14 +247,13 @@ objects:
           * Upon successful cluster creation, inform the user and provide the **cluster ID**.
 
           **Static Network Configuration**
-          * If they want static network configuration, you should first remind them of any existing static network configuration already present on the cluster by using the appropriate tool call. Show them the YAML only and not the mac_interface_map.
-          * Then generate the nmstate configuration for the desired hosts by asking them to describe the desired host network setup based **only** on the input to the generate_nmstate_yaml tool call. Don't make any assumptions about best or common practices unless told to.
-          * If the user does not provide interface names, use a reasonable default based on the type of the interface (e.g. for ethernet use eth0, eth1, etc). 
-          * Then use that tool call to generate an initial yaml and ask the user if they want to tweak anything. 
+          * If the user wants static network configuration, you should first remind them of any existing static network configuration already present on the cluster by using the appropriate tool call. Show them the YAML only and not the mac_interface_map.
+          * Then generate the nmstate configuration for the desired hosts by calling the proper tool. Don't make any assumptions about best or common practices unless told to.
+              * If the user does not provide interface names, use a reasonable default based on the type of the interface (e.g. for ethernet use eth0, eth1, etc).
+          * After generating the initial yaml ask the user if they want to tweak anything.
           * When modifying an existing host static network configuration, keep all existing configuration and only add or modify what the user explicitly asks for.
-          * If the user asks to change the generated yaml in a way not supported by the generate_nmstate_yaml tool call, attempt to alter the nmstate yaml yourself without making the tool call. 
-          * **Do not speak of the generate_nmstate_yaml or alter_static_network_config_nmstate_for_host tool call.**  
-          * After modifying nmstate yaml, validate it with the validate_nmstate_yaml tool call before presenting it to the user.
+          * If the user asks to change the generated yaml in a way not supported by the generate_nmstate_yaml tool call, attempt to alter the nmstate yaml yourself without making the tool call.
+          * After modifying nmstate yaml, validate it with the proper tool call before presenting it to the user.
           * **Always confirm the YAML with the user before applying it to the cluster.**
 
           **Mandatory Pre-Flight Checks for Cluster Creation**
@@ -268,6 +267,7 @@ objects:
       3.  **Host Discovery and Configuration:**
           * Once the Discovery ISO is generated, the user needs to boot hosts with it.
           * When a user indicates that hosts have been booted, first check for discovered hosts for that cluster and the cluster status.
+          * If no host were discovered indicate it to the user. Do not assume any hosts were discovered.
           * After hosts are discovered and appear in the hosts list, present the full list of discovered hosts to the user.
           * Proactively offer the next steps based on the cluster type:
               * **For a multi-node cluster:** Inform the user that roles can be automatically assigned or they can manually assign them. Offer to help with **manual role assignment** to a specific host (e.g., master, worker).

test/evals/eval_data.yaml

@@ -2,8 +2,8 @@
   conversation:
     - eval_id: basic_introduction
       eval_query: Hi!
-      eval_types: [response_eval:sub-string]
-      expected_keywords: ["OpenShift", "help"]
+      eval_types: [response_eval:accuracy]
+      expected_response: Hi! I'm the Red Hat OpenShift Lightspeed Intelligent Assistant, and I'm here to guide you through installing OpenShift using the Assisted Installer.
       description: Basic greeting test using keyword matching for reliability (avoids LLM judge flapping)
 
 - conversation_group: basic_cluster_request_conv
@@ -170,7 +170,7 @@
     - eval_id: supported_platforms_summary
       eval_query: Which platforms can you help me install on?
       eval_types: [response_eval:accuracy]
-      expected_response: I can help you install OpenShift on-premise using the Assisted Installer, either on bare metal servers or virtual machines (e.g., vSphere, KVM, libvirt). I do not support public cloud platforms like Amazon Web Services (AWS), Azure, or Google Cloud Platform (GCP).
+      expected_response: I can help you install OpenShift on-premise using the Assisted Installer, either on bare metal servers or virtual machines (VMs) like vSphere, KVM or libvirt. I do not support public cloud platforms like Amazon Web Services (AWS), Azure, or Google Cloud Platform (GCP).
       description: Ensure the assistant concisely states supported on-prem baremetal/VM scope and references Assisted Installer
 
 - conversation_group: cluster_id_from_name
@@ -215,7 +215,7 @@
         - - tool_name: generate_nmstate_yaml
             arguments: 
               params: |-
-                \{"ethernet_ifaces": \[\{"mac_address": "c5:d6:bc:f0:05:20", "name": "eth0"}\], "vlan_ifaces": \[{"name": "vlan0", "vlan_id": 400, "base_interface_name": "eth0", "ipv4_address": {"address": "10.0.0.5", "cidr_length": 24}}\], "dns": {"dns_servers": \["8.8.8.8"\]}}
+                (?s)^(?=.*"ethernet_ifaces":\s*\[\s*\{(?=.*"mac_address":\s*"c5:d6:bc:f0:05:20")(?=.*"name":\s*"eth0").*?\}\s*\])(?=.*"vlan_ifaces":\s*\[\s*\{(?=.*?"vlan_id":\s*400\b)(?=.*?"name":\s*"vlan0")(?=.*?"base_interface_name":\s*"eth0")(?=.*?"ipv4_address":\s*\{(?=.*?"address":\s*"10\.0\.0\.5")(?=.*?"cidr_length":\s*24\b).*?\}).*?\}\s*\])(?=.*"dns":\s*\{\s*"dns_servers":\s*\[\s*"8\.8\.8\.8"\s*\]\s*\}).*$
     - eval_id: apply_to_cluster
       eval_query: Yes, apply it to the cluster you just created.
       eval_types: [tool_eval]