[Infrastructure UI] Host filtering controls with url state by jennypavlova · Pull Request #2 · jennypavlova/kibana

jennypavlova · 2022-11-24T17:40:09Z

Workaround for persisting control filters panels

## Summary Fixes elastic#144161 As discussed [here](elastic#144161 (comment)), the existing implementation of update tags doesn't work well with real agents, as there are many conflicts with checkin, even when trying to add/remove one tag. Refactored the logic to make retries more efficient: - Instead of aborting the whole bulk action on conflicts, changed the conflict strategy to 'proceed'. This means, if an action of 50k agents has 1k conflicts, not all 50k is retried, but only the 1k conflicts, this makes it less likely to conflict on retry. - Because of this, on retry we have to know which agents don't yet have the tag added/removed. For this, added an additional filter to the `updateByQuery` request. Only adding the filter if there is exactly one `tagsToAdd` or one `tagsToRemove`. This is the main use case from the UI, and handling other cases would complicate the logic more (each additional tag to add/remove would result in another OR query, which would match more agents, making conflicts more likely). - Added this additional query on the initial request as well (not only retries) to save on unnecessary work e.g. if the user tries to add a tag on 50k agents, but 48k already have it, it is enough to update the remaining 2k agents. - This improvement has the effect that 'Agent activity' shows the real updated agent count, not the total selected. I think this is not really a problem for update tags. - Cleaned up some of the UI logic, because the conflicts are fully handled now on the backend. - Locally I couldn't reproduce the conflict with agent checkins, even with 1k horde agents. I'll try to test in cloud with more real agents. To verify: - Enroll 50k agents (I used 50k with create_agents script, and 1k with horde). Enroll 50k with horde if possible. - Select all on UI and try to add/remove one or more tags - Expect the changes to propagate quickly (up to 1m). It might take a few refreshes to see the result on agent list and tags list, because the UI polls the agents every 30s. It is expected that the tags list temporarily shows incorrect data because the action is async. E.g. removed `test3` tag and added `add` tag quickly: <img width="1776" alt="image" src="https://user-images.githubusercontent.com/90178898/207824481-411f0f70-d7e8-42a6-b73f-ed80e77b7700.png"> <img width="422" alt="image" src="https://user-images.githubusercontent.com/90178898/207824550-582d43fc-87db-45e1-ba58-15915447fefd.png"> The logs show the details of how many `version_conflicts` were there, and it decreased with retries. ``` [2022-12-15T10:32:12.937+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd541-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:12.981+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:16.477+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 29e9da70-7194-4e52-8004-2c1b19f6dfd5, total agents: 52000 [2022-12-15T10:32:16.537+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:22.893+01:00][DEBUG][plugins.fleet] {"took":9886,"timed_out":false,"total":52000,"updated":41143,"deleted":0,"batches":52,"version_conflicts":10857,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:26.066+01:00][DEBUG][plugins.fleet] {"took":9518,"timed_out":false,"total":52000,"updated":25755,"deleted":0,"batches":52,"version_conflicts":26245,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:27.401+01:00][ERROR][plugins.fleet] Action failed: version conflict of 10857 agents [2022-12-15T10:32:27.461+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:27.462+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:29.274+01:00][ERROR][plugins.fleet] Action failed: version conflict of 26245 agents [2022-12-15T10:32:29.353+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:29.353+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:31.480+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:31.481+01:00][DEBUG][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:31.481+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd541-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:31.481+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:31.485+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:33.841+01:00][DEBUG][plugins.fleet] {"took":2347,"timed_out":false,"total":10857,"updated":9857,"deleted":0,"batches":11,"version_conflicts":1000,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:34.556+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:34.557+01:00][DEBUG][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:34.557+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 29e9da70-7194-4e52-8004-2c1b19f6dfd5, total agents: 52000 [2022-12-15T10:32:34.557+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:34.560+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:35.388+01:00][ERROR][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de failed: version conflict of 1000 agents [2022-12-15T10:32:35.468+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:35.468+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de {"took":5509,"timed_out":false,"total":26245,"updated":26245,"deleted":0,"batches":27,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:42.722+01:00][INFO ][plugins.fleet] processed 26245 agents, took 5509ms [2022-12-15T10:32:42.723+01:00][INFO ][plugins.fleet] Removing task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:46.705+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:46.706+01:00][DEBUG][plugins.fleet] Retry #2 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:46.707+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd541-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:46.707+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:46.711+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:47.099+01:00][DEBUG][plugins.fleet] {"took":379,"timed_out":false,"total":1000,"updated":1000,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:47.623+01:00][INFO ][plugins.fleet] processed 1000 agents, took 379ms [2022-12-15T10:32:47.623+01:00][INFO ][plugins.fleet] Removing task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

…lastic#159352) ## Summary Skip `Security Solution Tests #2 / rule snoozing Rule editing page / actions tab adds an action to a snoozed rule` [This test failed on `main` as soon as it was merged.](https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/2952) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

… integration for ES|QL query generation via ELSER (elastic#167097) ## [Security Solution] [Elastic AI Assistant] LangChain Agents and Tools integration for ES|QL query generation via ELSER This PR integrates [LangChain](https://www.langchain.com/) [Agents](https://js.langchain.com/docs/modules/agents/) and [Tools](https://js.langchain.com/docs/modules/agents/tools/) with the [Elastic AI Assistant](https://www.elastic.co/blog/introducing-elastic-ai-assistant). These abstractions enable the LLM to dynamically choose whether or not to query, via [ELSER](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html), an [ES|QL](https://www.elastic.co/blog/elasticsearch-query-language-esql) knowledge base. Context from the knowledge base is used to generate `ES|QL` queries, or answer questions about `ES|QL`. Registration of the tool occurs in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts`: ```typescript const tools: Tool[] = [ new ChainTool({ name: 'esql-language-knowledge-base', description: 'Call this for knowledge on how to build an ESQL query, or answer questions about the ES|QL query language.', chain, }), ]; ``` The `tools` array above may be updated in future PRs to include, for example, an `ES|QL` query validator endpoint. ### Details The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts`: 1. Creates a `RetrievalQAChain` from an `ELSER` backed `ElasticsearchStore`, which serves as a knowledge base for `ES|QL`: ```typescript // ELSER backed ElasticsearchStore for Knowledge Base const esStore = new ElasticsearchStore(esClient, KNOWLEDGE_BASE_INDEX_PATTERN, logger); const chain = RetrievalQAChain.fromLLM(llm, esStore.asRetriever()); ``` 2. Registers the chain as a tool, which may be invoked by the LLM based on its description: ```typescript const tools: Tool[] = [ new ChainTool({ name: 'esql-language-knowledge-base', description: 'Call this for knowledge on how to build an ESQL query, or answer questions about the ES|QL query language.', chain, }), ]; ``` 3. Creates an Agent executor that combines the `tools` above, the `ActionsClientLlm` (an abstraction that calls `actionsClient.execute`), and memory of the previous messages in the conversation: ```typescript const executor = await initializeAgentExecutorWithOptions(tools, llm, { agentType: 'chat-conversational-react-description', memory, verbose: false, }); ``` Note: Set `verbose` above to `true` to for detailed debugging output from LangChain. 4. Calls the `executor`, kicking it off with `latestMessage`: ```typescript await executor.call({ input: latestMessage[0].content }); ``` ### Changes to `x-pack/packages/kbn-elastic-assistant` A client side change was required to the assistant, because the response returned from the agent executor is JSON. This response is parsed on the client in `x-pack/packages/kbn-elastic-assistant/impl/assistant/api.tsx`: ```typescript return assistantLangChain ? getFormattedMessageContent(result) : result; ``` Client-side parsing of the response only happens when then `assistantLangChain` feature flag is `true`. ## Desk testing Set ```typescript assistantLangChain={true} ``` in `x-pack/plugins/security_solution/public/assistant/provider.tsx` to enable this experimental feature in development environments. Also (optionally) set `verbose` to `true` in the following code in ``x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts``: ```typescript const executor = await initializeAgentExecutorWithOptions(tools, llm, { agentType: 'chat-conversational-react-description', memory, verbose: true, }); ``` After setting the feature flag and optionally enabling verbose debugging output, you may ask the assistant to generate an `ES|QL` query, per the example in the next section. ### Example output When the Elastic AI Assistant is asked: ``` From employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. "September 2019". Only show the query ``` it replies: ``` Here is the query to get the employee number and the formatted hire date for the 5 earliest employees by hire_date: FROM employees | KEEP emp_no, hire_date | EVAL month_year = DATE_FORMAT(hire_date, "MMMM YYYY") | SORT hire_date | LIMIT 5 ``` Per the screenshot below: ![ESQL_query_via_langchain_agents_and_tools](https://github.com/elastic/kibana/assets/4459398/c5cc75da-f7aa-4a12-9078-ed531f3463e7) The `verbose: true` output from LangChain logged to the console reveals that the prompt sent to the LLM includes text like the following: ``` Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\nesql-language-knowledge-base: Call this for knowledge on how to build an ESQL query, or answer questions about the ES|QL query language. ``` along with instructions for "calling" the tool like a function. The debugging output also reveals the agent selecting the tool, and returning results from ESLR: ``` [agent/action] [1:chain:AgentExecutor] Agent selected action: { "tool": "esql-language-knowledge-base", "toolInput": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.", "log": "```json\n{\n \"action\": \"esql-language-knowledge-base\",\n \"action_input\": \"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\"\n}\n```" } [tool/start] [1:chain:AgentExecutor > 4:tool:ChainTool] Entering Tool run with input: "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." [chain/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain] Entering Chain run with input: { "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." } [retriever/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 6:retriever:VectorStoreRetriever] Entering Retriever run with input: { "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." } [retriever/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 6:retriever:VectorStoreRetriever] [115ms] Exiting Retriever run with output: { "documents": [ { "pageContent": "[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n", ``` The documents containing `ES|QL` examples, retrieved from ELSER, are sent back to the LLM to answer the original question, per the abridged output below: ``` [llm/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain > 8:chain:LLMChain > 9:llm:ActionsClientLlm] Entering LLM run with input: { "prompts": [ "Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n\n[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n\n\n[[esql-date_trunc]]\n=== `DATE_TRUNC`\nRounds down a date to the closest interval. Intervals can be expressed using the\n<<esql-timespan-literals,timespan literal syntax>>.\n\n[source,esql]\n----\nFROM employees\n| EVAL year_hired = DATE_TRUNC(1 year, hire_date)\n| STATS count(emp_no) BY year_hired\n| SORT year_hired\n----\n\n\n[[esql-from]]\n=== `FROM`\n\nThe `FROM` source command returns a table with up to 10,000 documents from a\ndata stream, index, ``` ### Complete (verbose) LangChain output from the example The following `verbose: true` output from LangChain below was produced via the example in the previous section: ``` [chain/start] [1:chain:AgentExecutor] Entering Chain run with input: { "input": "\n\n\n\nFrom employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. \"September 2019\". Only show the query", "chat_history": [] } [chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input: { "input": "\n\n\n\nFrom employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. \"September 2019\". Only show the query", "chat_history": [], "agent_scratchpad": [], "stop": [ "Observation:" ] } [llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ActionsClientLlm] Entering LLM run with input: { "prompts": [ "[{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"SystemMessage\"],\"kwargs\":{\"content\":\"Assistant is a large language model trained by OpenAI.\\n\\nAssistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\\n\\nAssistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\\n\\nOverall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist. However, above all else, all responses must adhere to the format of RESPONSE FORMAT INSTRUCTIONS.\",\"additional_kwargs\":{}}},{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"HumanMessage\"],\"kwargs\":{\"content\":\"TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\nesql-language-knowledge-base: Call this for knowledge on how to build an ESQL query, or answer questions about the ES|QL query language.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nOutput a JSON markdown code snippet containing a valid JSON object in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{\\n \\\"action\\\": string, // The action to take. Must be one of [esql-language-knowledge-base]\\n \\\"action_input\\\": string // The input to the action. May be a stringified object.\\n}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly and conversationally to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{\\n \\\"action\\\": \\\"Final Answer\\\",\\n \\\"action_input\\\": string // You should put what you want to return to use here and make sure to use valid json newline characters.\\n}\\n```\\n\\nFor both options, remember to always include the surrounding markdown code snippet delimiters (begin with \\\"```json\\\" and end with \\\"```\\\")!\\n\\n\\nUSER'S INPUT\\n--------------------\\nHere is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n\\n\\n\\n\\nFrom employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. \\\"September 2019\\\". Only show the query\",\"additional_kwargs\":{}}}]" ] } [llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ActionsClientLlm] [3.08s] Exiting LLM run with output: { "generations": [ [ { "text": "```json\n{\n \"action\": \"esql-language-knowledge-base\",\n \"action_input\": \"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\"\n}\n```" } ] ] } [chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [3.09s] Exiting Chain run with output: { "text": "```json\n{\n \"action\": \"esql-language-knowledge-base\",\n \"action_input\": \"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\"\n}\n```" } [agent/action] [1:chain:AgentExecutor] Agent selected action: { "tool": "esql-language-knowledge-base", "toolInput": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.", "log": "```json\n{\n \"action\": \"esql-language-knowledge-base\",\n \"action_input\": \"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\"\n}\n```" } [tool/start] [1:chain:AgentExecutor > 4:tool:ChainTool] Entering Tool run with input: "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." [chain/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain] Entering Chain run with input: { "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." } [retriever/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 6:retriever:VectorStoreRetriever] Entering Retriever run with input: { "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." } [retriever/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 6:retriever:VectorStoreRetriever] [115ms] Exiting Retriever run with output: { "documents": [ { "pageContent": "[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/functions/date_format.asciidoc" } }, { "pageContent": "[[esql-date_trunc]]\n=== `DATE_TRUNC`\nRounds down a date to the closest interval. Intervals can be expressed using the\n<<esql-timespan-literals,timespan literal syntax>>.\n\n[source,esql]\n----\nFROM employees\n| EVAL year_hired = DATE_TRUNC(1 year, hire_date)\n| STATS count(emp_no) BY year_hired\n| SORT year_hired\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/functions/date_trunc.asciidoc" } }, { "pageContent": "[[esql-from]]\n=== `FROM`\n\nThe `FROM` source command returns a table with up to 10,000 documents from a\ndata stream, index, or alias. Each row in the resulting table represents a\ndocument. Each column corresponds to a field, and can be accessed by the name\nof that field.\n\n[source,esql]\n----\nFROM employees\n----\n\nYou can use <<api-date-math-index-names,date math>> to refer to indices, aliases\nand data streams. This can be useful for time series data, for example to access\ntoday's index:\n\n[source,esql]\n----\nFROM <logs-{now/d}>\n----\n\nUse comma-separated lists or wildcards to query multiple data streams, indices,\nor aliases:\n\n[source,esql]\n----\nFROM employees-00001,employees-*\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/source_commands/from.asciidoc" } }, { "pageContent": "[[esql-where]]\n=== `WHERE`\n\nUse `WHERE` to produce a table that contains all the rows from the input table\nfor which the provided condition evaluates to `true`:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=where]\n----\n\nWhich, if `still_hired` is a boolean field, can be simplified to:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereBoolean]\n----\n\n[discrete]\n==== Operators\n\nRefer to <<esql-operators>> for an overview of the supported operators.\n\n[discrete]\n==== Functions\n`WHERE` supports various functions for calculating values. Refer to\n<<esql-functions,Functions>> for more information.\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereFunction]\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/processing_commands/where.asciidoc" } } ] } [chain/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain] Entering Chain run with input: { "question": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.", "input_documents": [ { "pageContent": "[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/functions/date_format.asciidoc" } }, { "pageContent": "[[esql-date_trunc]]\n=== `DATE_TRUNC`\nRounds down a date to the closest interval. Intervals can be expressed using the\n<<esql-timespan-literals,timespan literal syntax>>.\n\n[source,esql]\n----\nFROM employees\n| EVAL year_hired = DATE_TRUNC(1 year, hire_date)\n| STATS count(emp_no) BY year_hired\n| SORT year_hired\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/functions/date_trunc.asciidoc" } }, { "pageContent": "[[esql-from]]\n=== `FROM`\n\nThe `FROM` source command returns a table with up to 10,000 documents from a\ndata stream, index, or alias. Each row in the resulting table represents a\ndocument. Each column corresponds to a field, and can be accessed by the name\nof that field.\n\n[source,esql]\n----\nFROM employees\n----\n\nYou can use <<api-date-math-index-names,date math>> to refer to indices, aliases\nand data streams. This can be useful for time series data, for example to access\ntoday's index:\n\n[source,esql]\n----\nFROM <logs-{now/d}>\n----\n\nUse comma-separated lists or wildcards to query multiple data streams, indices,\nor aliases:\n\n[source,esql]\n----\nFROM employees-00001,employees-*\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/source_commands/from.asciidoc" } }, { "pageContent": "[[esql-where]]\n=== `WHERE`\n\nUse `WHERE` to produce a table that contains all the rows from the input table\nfor which the provided condition evaluates to `true`:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=where]\n----\n\nWhich, if `still_hired` is a boolean field, can be simplified to:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereBoolean]\n----\n\n[discrete]\n==== Operators\n\nRefer to <<esql-operators>> for an overview of the supported operators.\n\n[discrete]\n==== Functions\n`WHERE` supports various functions for calculating values. Refer to\n<<esql-functions,Functions>> for more information.\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereFunction]\n----\n", "metadata": { "source": "/Users/andrew.goldstein/Projects/forks/spong/kibana/x-pack/plugins/elastic_assistant/server/knowledge_base/esql/docs/processing_commands/where.asciidoc" } } ], "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'." } [chain/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain > 8:chain:LLMChain] Entering Chain run with input: { "question": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.", "query": "Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.", "context": "[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n\n\n[[esql-date_trunc]]\n=== `DATE_TRUNC`\nRounds down a date to the closest interval. Intervals can be expressed using the\n<<esql-timespan-literals,timespan literal syntax>>.\n\n[source,esql]\n----\nFROM employees\n| EVAL year_hired = DATE_TRUNC(1 year, hire_date)\n| STATS count(emp_no) BY year_hired\n| SORT year_hired\n----\n\n\n[[esql-from]]\n=== `FROM`\n\nThe `FROM` source command returns a table with up to 10,000 documents from a\ndata stream, index, or alias. Each row in the resulting table represents a\ndocument. Each column corresponds to a field, and can be accessed by the name\nof that field.\n\n[source,esql]\n----\nFROM employees\n----\n\nYou can use <<api-date-math-index-names,date math>> to refer to indices, aliases\nand data streams. This can be useful for time series data, for example to access\ntoday's index:\n\n[source,esql]\n----\nFROM <logs-{now/d}>\n----\n\nUse comma-separated lists or wildcards to query multiple data streams, indices,\nor aliases:\n\n[source,esql]\n----\nFROM employees-00001,employees-*\n----\n\n\n[[esql-where]]\n=== `WHERE`\n\nUse `WHERE` to produce a table that contains all the rows from the input table\nfor which the provided condition evaluates to `true`:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=where]\n----\n\nWhich, if `still_hired` is a boolean field, can be simplified to:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereBoolean]\n----\n\n[discrete]\n==== Operators\n\nRefer to <<esql-operators>> for an overview of the supported operators.\n\n[discrete]\n==== Functions\n`WHERE` supports various functions for calculating values. Refer to\n<<esql-functions,Functions>> for more information.\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereFunction]\n----\n" } [llm/start] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain > 8:chain:LLMChain > 9:llm:ActionsClientLlm] Entering LLM run with input: { "prompts": [ "Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n\n[[esql-date_format]]\n=== `DATE_FORMAT`\nReturns a string representation of a date in the provided format. If no format\nis specified, the `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format is used.\n\n[source,esql]\n----\nFROM employees\n| KEEP first_name, last_name, hire_date\n| EVAL hired = DATE_FORMAT(hire_date, \"YYYY-MM-dd\")\n----\n\n\n[[esql-date_trunc]]\n=== `DATE_TRUNC`\nRounds down a date to the closest interval. Intervals can be expressed using the\n<<esql-timespan-literals,timespan literal syntax>>.\n\n[source,esql]\n----\nFROM employees\n| EVAL year_hired = DATE_TRUNC(1 year, hire_date)\n| STATS count(emp_no) BY year_hired\n| SORT year_hired\n----\n\n\n[[esql-from]]\n=== `FROM`\n\nThe `FROM` source command returns a table with up to 10,000 documents from a\ndata stream, index, or alias. Each row in the resulting table represents a\ndocument. Each column corresponds to a field, and can be accessed by the name\nof that field.\n\n[source,esql]\n----\nFROM employees\n----\n\nYou can use <<api-date-math-index-names,date math>> to refer to indices, aliases\nand data streams. This can be useful for time series data, for example to access\ntoday's index:\n\n[source,esql]\n----\nFROM <logs-{now/d}>\n----\n\nUse comma-separated lists or wildcards to query multiple data streams, indices,\nor aliases:\n\n[source,esql]\n----\nFROM employees-00001,employees-*\n----\n\n\n[[esql-where]]\n=== `WHERE`\n\nUse `WHERE` to produce a table that contains all the rows from the input table\nfor which the provided condition evaluates to `true`:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=where]\n----\n\nWhich, if `still_hired` is a boolean field, can be simplified to:\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereBoolean]\n----\n\n[discrete]\n==== Operators\n\nRefer to <<esql-operators>> for an overview of the supported operators.\n\n[discrete]\n==== Functions\n`WHERE` supports various functions for calculating values. Refer to\n<<esql-functions,Functions>> for more information.\n\n[source,esql]\n----\ninclude::{esql-specs}/docs.csv-spec[tag=whereFunction]\n----\n\n\nQuestion: Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\nHelpful Answer:" ] } [llm/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain > 8:chain:LLMChain > 9:llm:ActionsClientLlm] [2.23s] Exiting LLM run with output: { "generations": [ [ { "text": "FROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5" } ] ] } [chain/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain > 8:chain:LLMChain] [2.23s] Exiting Chain run with output: { "text": "FROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5" } [chain/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain > 7:chain:StuffDocumentsChain] [2.23s] Exiting Chain run with output: { "text": "FROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5" } [chain/end] [1:chain:AgentExecutor > 4:tool:ChainTool > 5:chain:RetrievalQAChain] [2.35s] Exiting Chain run with output: { "text": "FROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5" } [tool/end] [1:chain:AgentExecutor > 4:tool:ChainTool] [2.35s] Exiting Tool run with output: "FROM employees | KEEP emp_no, hire_date | EVAL month_year = DATE_FORMAT(hire_date, "MMMM YYYY") | SORT hire_date | LIMIT 5" [chain/start] [1:chain:AgentExecutor > 10:chain:LLMChain] Entering Chain run with input: { "input": "\n\n\n\nFrom employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. \"September 2019\". Only show the query", "chat_history": [], "agent_scratchpad": [ { "lc": 1, "type": "constructor", "id": [ "langchain", "schema", "AIMessage" ], "kwargs": { "content": "```json\n{\n \"action\": \"esql-language-knowledge-base\",\n \"action_input\": \"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\"\n}\n```", "additional_kwargs": {} } }, { "lc": 1, "type": "constructor", "id": [ "langchain", "schema", "HumanMessage" ], "kwargs": { "content": "TOOL RESPONSE:\n---------------------\nFROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5\n\nUSER'S INPUT\n--------------------\n\nOkay, so what is the response to my last comment? If using information obtained from the tools you must mention it explicitly without mentioning the tool names - I have forgotten all TOOL RESPONSES! Remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else.", "additional_kwargs": {} } } ], "stop": [ "Observation:" ] } [llm/start] [1:chain:AgentExecutor > 10:chain:LLMChain > 11:llm:ActionsClientLlm] Entering LLM run with input: { "prompts": [ "[{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"SystemMessage\"],\"kwargs\":{\"content\":\"Assistant is a large language model trained by OpenAI.\\n\\nAssistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\\n\\nAssistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\\n\\nOverall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist. However, above all else, all responses must adhere to the format of RESPONSE FORMAT INSTRUCTIONS.\",\"additional_kwargs\":{}}},{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"HumanMessage\"],\"kwargs\":{\"content\":\"TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\nesql-language-knowledge-base: Call this for knowledge on how to build an ESQL query, or answer questions about the ES|QL query language.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nOutput a JSON markdown code snippet containing a valid JSON object in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{\\n \\\"action\\\": string, // The action to take. Must be one of [esql-language-knowledge-base]\\n \\\"action_input\\\": string // The input to the action. May be a stringified object.\\n}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly and conversationally to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{\\n \\\"action\\\": \\\"Final Answer\\\",\\n \\\"action_input\\\": string // You should put what you want to return to use here and make sure to use valid json newline characters.\\n}\\n```\\n\\nFor both options, remember to always include the surrounding markdown code snippet delimiters (begin with \\\"```json\\\" and end with \\\"```\\\")!\\n\\n\\nUSER'S INPUT\\n--------------------\\nHere is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n\\n\\n\\n\\nFrom employees, I want to see the 5 earliest employees (hire_date), I want to display only the month and the year that they were hired in and their employee number (emp_no). Format the date as e.g. \\\"September 2019\\\". Only show the query\",\"additional_kwargs\":{}}},{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"AIMessage\"],\"kwargs\":{\"content\":\"```json\\n{\\n \\\"action\\\": \\\"esql-language-knowledge-base\\\",\\n \\\"action_input\\\": \\\"Display the 'emp_no', month and year of the 5 earliest employees by 'hire_date'. Format the date as 'Month Year'.\\\"\\n}\\n```\",\"additional_kwargs\":{}}},{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain\",\"schema\",\"HumanMessage\"],\"kwargs\":{\"content\":\"TOOL RESPONSE:\\n---------------------\\nFROM employees\\n| KEEP emp_no, hire_date\\n| EVAL month_year = DATE_FORMAT(hire_date, \\\"MMMM YYYY\\\")\\n| SORT hire_date\\n| LIMIT 5\\n\\nUSER'S INPUT\\n--------------------\\n\\nOkay, so what is the response to my last comment? If using information obtained from the tools you must mention it explicitly without mentioning the tool names - I have forgotten all TOOL RESPONSES! Remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else.\",\"additional_kwargs\":{}}}]" ] } [llm/end] [1:chain:AgentExecutor > 10:chain:LLMChain > 11:llm:ActionsClientLlm] [6.47s] Exiting LLM run with output: { "generations": [ [ { "text": "```json\n{\n \"action\": \"Final Answer\",\n \"action_input\": \"Here is the query to get the employee number and the formatted hire date for the 5 earliest employees by hire_date:\\n\\nFROM employees\\n| KEEP emp_no, hire_date\\n| EVAL month_year = DATE_FORMAT(hire_date, \\\"MMMM YYYY\\\")\\n| SORT hire_date\\n| LIMIT 5\"\n}\n```" } ] ] } [chain/end] [1:chain:AgentExecutor > 10:chain:LLMChain] [6.47s] Exiting Chain run with output: { "text": "```json\n{\n \"action\": \"Final Answer\",\n \"action_input\": \"Here is the query to get the employee number and the formatted hire date for the 5 earliest employees by hire_date:\\n\\nFROM employees\\n| KEEP emp_no, hire_date\\n| EVAL month_year = DATE_FORMAT(hire_date, \\\"MMMM YYYY\\\")\\n| SORT hire_date\\n| LIMIT 5\"\n}\n```" } [chain/end] [1:chain:AgentExecutor] [11.91s] Exiting Chain run with output: { "output": "Here is the query to get the employee number and the formatted hire date for the 5 earliest employees by hire_date:\n\nFROM employees\n| KEEP emp_no, hire_date\n| EVAL month_year = DATE_FORMAT(hire_date, \"MMMM YYYY\")\n| SORT hire_date\n| LIMIT 5" } ```

## Summary ### This PR enables user roles testing in FTR We use SAML authentication to get session cookie for user with the specific role. The cookie is cached on FTR service side so we only make SAML auth one time per user within FTR config run. For Kibana CI service relies on changes coming in elastic#170852 In order to run FTR tests locally against existing MKI project: - add `.ftr/role_users.json` in Kibana root dir ``` { "viewer": { "email": "...", "password": "..." }, "developer": { "email": "...", "password": "..." } } ``` - set Cloud hostname (!not project hostname!) with TEST_CLOUD_HOST_NAME, e.g. `export TEST_CLOUD_HOST_NAME=console.qa.cld.elstc.co` ### How to use: - functional tests: ``` const svlCommonPage = getPageObject('svlCommonPage'); before(async () => { // login with Viewer role await svlCommonPage.loginWithRole('viewer'); // you are logged in in browser and on project home page, start the test }); it('has project header', async () => { await svlCommonPage.assertProjectHeaderExists(); }); ``` - API integration tests: ``` const svlUserManager = getService('svlUserManager'); const supertestWithoutAuth = getService('supertestWithoutAuth'); let credentials: { Cookie: string }; before(async () => { // get auth header for Viewer role credentials = await svlUserManager.getApiCredentialsForRole('viewer'); }); it('returns full status payload for authenticated request', async () => { const { body } = await supertestWithoutAuth .get('/api/status') .set(credentials) .set('kbn-xsrf', 'kibana'); expect(body.name).to.be.a('string'); expect(body.uuid).to.be.a('string'); expect(body.version.number).to.be.a('string'); }); ``` Flaky-test-runner: #1 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4081 #2 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4114 --------- Co-authored-by: Robert Oskamp <traeluki@gmail.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

## Summary ![esql_syntax_fix](https://github.com/elastic/kibana/assets/924948/263e8cfd-3325-4a28-903f-d55dbb3dfeae)

## Summary The previous PR elastic#161813 was reverted due to the broken webpack config elastic@eef1afc --------- Co-authored-by: Tiago Costa <tiago.costa@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jon <jon@elastic.co>

…ic#175194) ## Summary This PR fixes the issue causing (mostly) [login journey](https://buildkite.com/elastic/kibana-single-user-performance/builds/12398#018d1149-cc2e-4591-a61c-176768081e2c) stuck for 14 min waiting for Telemetry call response. <img width="964" alt="Screenshot 2024-01-22 at 11 12 24" src="https://github.com/elastic/kibana/assets/10977896/8cadc2ec-ee84-42f6-8a0c-ad949367429c"> I believe the issue was in how we handle the Observables for request events. I added extra comment in the particular code change. I no longer can reproduce it, all the events are reported correctly: <img width="964" alt="image" src="https://github.com/elastic/kibana/assets/10977896/fa2c4b27-dcf2-480b-a07f-aeb23045149a"> Logs cleaning is to log in console only performance metrics event but not all EBT elements. Also not to report some browser errors that not Kibana specific. Testing: run the following script 3-4 times ``` PERFORMANCE_ENABLE_TELEMETRY=1 node scripts/run_performance.js --journey-path x-pack/performance/journeys/login.ts ``` - script is completed without delays (e.g. doesn't hang on after hook in TEST phase) - telemetry requests are logged with correct counter and all finished, e.g. `Waiting for telemetry request #2 to complete` is followed by `Telemetry request #2 complete` - only events started with `Report event "performance_metric"` are in console output

## Summary Set `security.session.cleanupInterval` to 5h for session concurrency test. ### **Prerequisites** - Task for session cleanup with [default schedule set to 1h](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security/server/config.ts#L222). - Task polling interval is set to [3000ms](https://github.com/elastic/kibana/blob/main/x-pack/plugins/task_manager/server/config.ts#L13). - We override `scheduledAt` once we make a request in [runCleanupTaskSoon](https://github.com/elastic/kibana/blob/main/x-pack/test/security_api_integration/tests/session_concurrent_limit/cleanup.ts#L145). ### **Hypothesis** Taking into consideration that: - `session_cleanup` task is not the only one scheduled during test run. - There is sort of an exponential backoff implemented for task polling if there are too many retries. - Clock jitter. I had a hypothesis that if our whole test run exceeds 1h or polling interval gets adjusted because of retries we might end up executing the scheduled cleanup before we trigger `runCleanupTaskSoon` (this is there we drop 1 session already). ### **FTR runs (x55 each)** - `cleanupInterval` set to 5h: [#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5986) :green_circle:, [#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5987) :green_circle: - `cleanupInterval` set to default 1h: [#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5983) :green_circle:, [#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5982) :red_circle: (has 2 failures out of 55) ### Checklist - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) __Fixes: https://github.com/elastic/kibana/issues/149091__

## Summary Resolves elastic#143905. This PR adds support for integration-level outputs. This means that different integrations within the same agent policy can now be configured to send data to different locations. This feature is gated behind `enterprise` level subscription. For each input, the agent policy will configure sending data to the following outputs in decreasing order of priority: 1. Output set specifically on the integration policy 2. Output set specifically on the integration's parent agent policy (including the case where an integration policy belongs to multiple agent policies) 3. Global default data output set via Fleet Settings Integration-level outputs will respect the same rules as agent policy-level outputs: - Certain integrations are disallowed from using certain output types, attempting to add them to each other via creation, updating, or "defaulting", will fail - `fleet-server`, `synthetics`, and `apm` can only use same-cluster Elasticsearch output - When an output is deleted, any integrations that were specifically using it will "clear" their output configuration and revert back to either `#2` or `#3` in the above list - When an output is edited, all agent policies across all spaces that use it will be bumped to a new revision, this includes: - Agent policies that have that output specifically set in their settings (existing behavior) - Agent policies that contain integrations which specifically has that output set (new behavior) - When a proxy is edited, the same new revision bump above will apply for any outputs using that proxy The final agent policy YAML that is generated will have: - `outputs` block that includes: - Data and monitoring outputs set at the agent policy level (existing behavior) - Any additional outputs set at the integration level, if they differ from the above - `outputs_permissions` block that includes permissions for each Elasticsearch output depending on which integrations and/or agent monitoring are assigned to it Integration policies table now includes `Output` column. If the output is defaulting to agent policy-level output, or global setting output, a tooltip is shown: <img width="1392" alt="image" src="https://github.com/user-attachments/assets/5534716b-49b5-402a-aa4a-4ba6533e0ca8"> Configuring an integration-level output is done under Advanced options in the policy editor. Setting to the blank value will "clear" the output configuration. The list of available outputs is filtered by what outputs are available for that integration (see above): <img width="799" alt="image" src="https://github.com/user-attachments/assets/617af6f4-e8f8-40b1-b476-848f8ac96e76"> An example of failure: ES output cannot be changed to Kafka while there is an integration <img width="1289" alt="image" src="https://github.com/user-attachments/assets/11847eb5-fd5d-4271-8464-983d7ab39218"> ## TODO - [x] Adjust side effects of editing/deleting output when policies use it across different spaces - [x] Add API integration tests - [x] Update OpenAPI spec - [x] Create doc issue ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…193441) ## Summary More files to be regenerated with a different shape since the js-yaml update: elastic#190678

…tion-into-per-data-structure-serialization-logic added span-link normalization and fixed errors in getTransaction

Mainly to address timeouts on [Investigations - Security Solution Cypress Tests #2](https://buildkite.com/elastic/kibana-on-merge/builds/58405#01942904-03f4-406a-856d-d1f24d6fd533) but made a pass on all groups. Timings were based on the on-merge pipeline https://buildkite.com/elastic/kibana-on-merge.

…) (elastic#222641) # Backport This will backport the following commits from `main` to `8.17`: - [Update dependency whatwg-fetch to ^3.6.20 (main) (#206179)](https://github.com/elastic/kibana/pull/206179)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Brad White <brad.white@elastic.co>

…) (elastic#222643) # Backport This will backport the following commits from `main` to `8.19`: - [Update dependency whatwg-fetch to ^3.6.20 (main) (#206179)](https://github.com/elastic/kibana/pull/206179)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Brad White <brad.white@elastic.co>

…) (elastic#222642) # Backport This will backport the following commits from `main` to `8.18`: - [Update dependency whatwg-fetch to ^3.6.20 (main) (#206179)](https://github.com/elastic/kibana/pull/206179)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Brad White <brad.white@elastic.co>

… (elastic#222644) # Backport This will backport the following commits from `main` to `9.0`: - [Update dependency whatwg-fetch to ^3.6.20 (main) (#206179)](https://github.com/elastic/kibana/pull/206179)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Brad White <brad.white@elastic.co>

…gent is lost (elastic#237705) ## Summary Since we use spot agents for our FTR tests, there are cases that some of the configs are already run but due to agent lost, we will try all the config of that group again. In this PR, we use Buildkite metadata config to keep track of configs that are already executed, so that if the agent is lost, we first check if there is a metadata for this config, if yes, we will skip running that config. For this logic to work, we also need to save the Scout events related to each config after running each config instead of at the end of each config group, to ensure if the agent is lost, we will keep the related execution stats and events. #### Expected improvement |Build|Before|After|Improvement| |---|---|---|---| |[348415](https://buildkite.com/elastic/kibana-pull-request/builds/348415)|2h 25m (estimate)|1h 37m|saves 48 minutes (~33% faster)| |[348229](https://buildkite.com/elastic/kibana-pull-request/builds/348229)|1h 39m|1h 10m (estimate)|saves 29 minutes (~30% faster)| |[348223](https://buildkite.com/elastic/kibana-pull-request/builds/348223/waterfall)|2h 3m|1h 17m (estimate)| saves 46 minutes (~37% faster)| In the last [example](https://buildkite.com/elastic/kibana-pull-request/builds/348223/waterfall), `FTR Configs #2` takes almost double the time because the agent is lost when executing the last config. <img width="2588" height="456" alt="image" src="https://github.com/user-attachments/assets/992ffc6b-4412-47f9-9dd2-ecd5ff607358" /> Here is a video that illustrates the issue for this [build](https://buildkite.com/elastic/kibana-pull-request/builds/348229): https://github.com/user-attachments/assets/5f499f78-5841-40e7-8582-e761b885ed41 ### 🧪 How to test What I did was run a small portion of the tests in this [build](https://buildkite.com/elastic/kibana-pull-request/builds/348316), wait for one config to finish and report its stats, then cancel the build and retry it to see if the new build would skip the completed config as expected. In this [build](https://buildkite.com/elastic/kibana-pull-request/builds/348415), it also improved `FTR Config 6`, although the previous failure was "Exited with status 10" not agent loss. - **Before**: 2h 25m (estimate) - **After**: 1h 37m - **Time saved**: 48 minutes (~33% faster) <img width="2934" height="146" alt="image" src="https://github.com/user-attachments/assets/88b7ad5a-46b1-42ad-9321-f33a81d89ee6" />

…re (#2… (elastic#229752) Backport elastic#224393 to 8.19 --------- Co-authored-by: Cristina Amico <criamico@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…#258775) ## Summary Creates the foundation package `@kbn/evals-extensions` for advanced evaluation capabilities. This package will house features ported from cursor-plugin-evals and serve as the home for Phases 3-5 of the evals roadmap. ## Architecture **One-way dependency:** - ✅ kbn-evals-extensions depends on kbn-evals - ❌ kbn-evals has NO dependency on kbn-evals-extensions Evaluation suites opt-in by importing from extensions directly. ## What's Included ✅ Package structure and build configuration ✅ Comprehensive documentation ✅ 5 passing unit tests ✅ CODEOWNERS entry ✅ No functional changes ## Validation ✅ Bootstrap, type check, tests, eslint, check_changes.ts all passed ✅ No circular dependencies ## Roadmap This enables PRs #2-10 for cost tracking, dataset management, safety evaluators, UI components, DX enhancements, analytics, A/B testing, human-in-the-loop, and IDE integration. ## Related - Part of elastic#257821 - Enables elastic#257823, elastic#257824, elastic#257825, elastic#257826 - Addresses elastic#255820 Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Garrett Spong <garrett.spong@elastic.co>

@timestamp

Closes elastic#258318 Closes elastic#258319 ## Summary Adds logic to the alert episodes table to display `.alert_actions` information. This includes: - New action-specific API paths. - Snooze - **Per group hash.** - Button in the actions column opens a popover where an `until` can be picked. - **When snoozed** - A bell shows up in the status column. - Mouse over the bell icon to see until when the snooze is in effect. - Unsnooze - **Per group hash.** - Clicking the button removes the snooze. - Ack/Unack - **Per episode.** - Button in the actions column - When "acked", an icon shows in the status column. - Tags - This PR only handles displaying tags. They need to be created via API. - Resolve/Unresolve - **Per group hash.** - Button inside the ellipsis always - The status is turned to `inactive` **regardless of the "real" status.** <img width="1704" height="672" alt="Screenshot 2026-03-25 at 16 04 12" src="https://github.com/user-attachments/assets/5ef4111a-6e0c-4114-a60e-ce5f81a86ac6" /> ## Testing <details> <summary>POST mock episodes</summary> ``` POST _bulk { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:00:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:01:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:02:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:03:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:04:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:05:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:06:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:07:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:08:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:09:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:10:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:11:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:12:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:13:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:19:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:22:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:26:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "gh-7", "episode": { "id": "ep-007", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "gh-7", "episode": { "id": "ep-007", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "gh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "gh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "gh-10", "episode": { "id": "ep-010", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "gh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "gh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "no_data" } ``` </details> - In the POST above, episodes 1 and 3, and episodes 6 and 9 have the same group hashes. - Go to `https://localhost:5601/app/observability/alerts-v2` and try all buttons. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…260054) ## Summary issue: elastic/security-team#15982. (Resolves requirement #2) Enriches `SelectionContext` with step `values` so that `PropertySelectionHandler` implementations (`search`, `resolve`, `getDetails`) can access sibling property values from the current step definition. There are no existing steps using `context.values` yet — this is a prerequisite for upcoming steps from the Cases team that need to read sibling properties (e.g. `owner`) to scope their search/resolve logic. ### What changed **`SelectionContext.values`** — A new `values` field (`{ config, input }`) is populated from the step's YAML properties at the time `search`/`resolve`/`getDetails` are called. Handlers can now read sibling values like `context.values.input.owner` instead of having no visibility into the rest of the step. **YAML value extraction fix** — `getValueFromValueNode` now handles non-scalar YAML nodes (arrays/sequences) via `.toJSON()`, fixing a bug where array properties like `owner: [securitySolution]` appeared as `undefined` in `context.values`. **Example** — An example has been implemented in the `examples.externalStep` from the _workflows_examples_ plugin (test with `node scripts/kibana --dev --run-examples`) ### Demo https://github.com/user-attachments/assets/b0cf18ae-5906-4561-bf5e-31b228b08a30 ### Files changed | Area | Files | Change | |------|-------|--------| | Shared types | `kbn-workflows/types/v1.ts`, `latest.ts` | Added `StepSelectionValues` interface, `values` field on `SelectionContext`, generic type parameters on 4 interfaces | | Step registry | `workflows_extensions/.../step_registry/types.ts` | Thread `Config`/`Input` schema types into `StepPropertyHandler` for automatic inference | | Value builder | `build_workflow_lookup.ts` | Added `buildStepSelectionValues()` helper + fixed `getValueFromValueNode` for non-scalar nodes | | Context wiring | `collect_all_custom_property_items.ts`, `get_custom_property_suggestions.ts` | Pass `values` when constructing `SelectionContext` | | Docs | `STEPS.md` | Updated `SelectionContext` type definition and documented `context.values` | | Tests | `build_workflow_lookup.test.ts`, `validate_custom_properties.test.ts`, `get_custom_property_suggestions.test.ts` | New tests for value building (including arrays), updated assertions for new context shape | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

jennypavlova force-pushed the 140445-host-filtering-controls-with-url-state branch 2 times, most recently from e8c1a1f to e9677b6 Compare November 24, 2022 17:46

Add control filters to url state

636f151

jennypavlova force-pushed the 140445-host-filtering-controls-with-url-state branch from e9677b6 to 636f151 Compare November 24, 2022 20:16

Add explicitInput types

cd7981a

jennypavlova merged commit 031e4f9 into 140445-host-filtering-controls Nov 24, 2022

jennypavlova pushed a commit that referenced this pull request Jan 3, 2024

[ES|QL] Fix highlight for multiline #2 (elastic#173827)

41375d7

## Summary ![esql_syntax_fix](https://github.com/elastic/kibana/assets/924948/263e8cfd-3325-4a28-903f-d55dbb3dfeae)

MiriamAparicio pushed a commit that referenced this pull request Sep 25, 2024

[fix] regenerate and add missing openapi-generated files #2 (elastic#…

f785002

…193441) ## Summary More files to be regenerated with a different shape since the js-yaml update: elastic#190678

jennypavlova pushed a commit that referenced this pull request Sep 30, 2024

Merge pull request #2 from bryce-b/192749-refactor-the-normalize-func…

b4c5a95

…tion-into-per-data-structure-serialization-logic added span-link normalization and fixed errors in getTransaction

jennypavlova pushed a commit that referenced this pull request Jul 2, 2025

Merge pull request #2 from crespocarlos/k8s-poc-fixes

367435a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Infrastructure UI] Host filtering controls with url state#2

[Infrastructure UI] Host filtering controls with url state#2
jennypavlova merged 2 commits into140445-host-filtering-controlsfrom
140445-host-filtering-controls-with-url-state

jennypavlova commented Nov 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jennypavlova commented Nov 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant