Reverts change to x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/prompts.ts by andrew-goldstein · Pull Request #7 · patrykkopycinski/kibana

andrew-goldstein · 2024-08-02T20:57:25Z

reverts change to x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/prompts.ts

…hain/graphs/default_assistant_graph/prompts.ts

fixes [#8](elastic/observability-accessibility#8) fixes [#7](elastic/observability-accessibility#7) ## Summary Fixes APM breadcrumbs on serverless | Serverless | Stateful | |---|---| | <img width="700px" alt="image" src="https://github.com/user-attachments/assets/944a7d58-7de3-4a7f-be02-3c8c1110a0e2"> |<img width="800px" alt="image" src="https://github.com/user-attachments/assets/450664b1-ddfc-4395-9fa3-a7b941affb3b">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/944a7d58-7de3-4a7f-be02-3c8c1110a0e2"> |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/450664b1-ddfc-4395-9fa3-a7b941affb3b">| | <img width="500px" alt="image" src="https://github.com/user-attachments/assets/944a7d58-7de3-4a7f-be02-3c8c1110a0e2"> |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/cb8a39e2-ca33-4cf9-a8ac-4c84566d092d">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/151a3a9c-c81e-4558-9d00-e695e3d1d79c">|<img width="500px" alt="image" src="https://github.com/user-attachments/assets/2562e96f-d5e4-4aa4-a221-6721f8995883">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/8d877d11-8c3f-4ac5-8146-6a11125eae7c">|<img width="500px" alt="image" src="https://github.com/user-attachments/assets/36e588cb-4c18-4d66-a2c6-f0e66392f708">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/14253196-06de-4343-811f-61aa31ea0d1e">|<img width="500px" alt="image" src="https://github.com/user-attachments/assets/0cdfc83f-6545-433f-8c14-5bbf2a581175">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/89a58e2b-2cef-4188-b2be-f359ba6890db">|<img width="500px" alt="image" src="https://github.com/user-attachments/assets/f15e767f-5b60-4485-ac71-7b6fd850ec50">| |<img width="500px" alt="image" src="https://github.com/user-attachments/assets/a0f7bfae-bfda-4f49-b92a-e736d80fea4c">|<img width="500px" alt="image" src="https://github.com/user-attachments/assets/680db8ab-58b8-454b-a0d7-6e1681dbe616">| ### How to test #### Serverless - Start a local ES serverless instance: `yarn es serverless --projectType=oblt --ssl -k/--insecure` - Start a local Kibana serverless instance: ` yarn start --serverless=oblt --no-ssl` - Run some synthtrace scenarios - `NODE_TLS_REJECT_UNAUTHORIZED=0 node scripts/synthtrace mobile.ts --live --target=https://elastic_serverless:changeme@127.0.0.1:9200 --kibana=http://elastic_serverless:changeme@0.0.0.0:5601` - `NODE_TLS_REJECT_UNAUTHORIZED=0 node scripts/synthtrace service_map.ts --live --target=https://elastic_serverless:changeme@127.0.0.1:9200 --kibana=http://elastic_serverless:changeme@0.0.0.0:5601` - Navigate to Applications and click through the links ### Stateful - Start a local ES and Kibana instance - Run the some synthtrace scenarios: - `node scripts/synthtrace mobile.ts --live` - `node scripts/synthtrace service_map.ts --live` - Navigate to Applications and click through the links --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

This establishes the structure for advanced evaluation capabilities ported from cursor-plugin-evals and serves as the home for Phases 3-5 of the evals roadmap. ## Architecture The package is designed to be completely independent from @kbn/evals: ``` Evaluation Suites ├──> @kbn/evals (core) └──> @kbn/evals-extensions (advanced features) └──> depends on @kbn/evals ``` **Dependency Rule:** - ✅ kbn-evals-extensions CAN import from kbn-evals - ❌ kbn-evals MUST NOT import from kbn-evals-extensions ## This PR **What's included:** - Package structure (package.json, kibana.jsonc, tsconfig.json) - Placeholder exports (no functional changes) - Test infrastructure (5 passing tests) - Comprehensive documentation **What's NOT included:** - No functional features (placeholder exports only) - No changes to @kbn/evals package - No changes to evaluation suite behavior ## Validation ✅ Bootstrap completed successfully ✅ Type check passed ✅ All tests passing (5/5) ✅ ESLint passed ✅ No circular dependencies ✅ check_changes.ts passed ## Roadmap This foundation enables parallel development of: - PR #2: Cost tracking & metadata enrichment - PR #3: Dataset management utilities - PR #4: Safety evaluators (toxicity, PII, bias, etc.) - PR #5: UI components (run comparison, example explorer) - PR #6: DX enhancements (watch mode, caching, parallel) - PR #7: Advanced analytics - PR #8: A/B testing & active learning - PR #9: Human-in-the-loop workflows - PR elastic#10: IDE integration ## Related Issues - Closes part of elastic#257821 (Epic: Extend @kbn/evals) - Enables elastic#257823 (Phase 2: CI Quality Gates) - Enables elastic#257824 (Phase 3: Red-Teaming) - Enables elastic#257825 (Phase 4: Lens Dashboards) - Enables elastic#257826 (Phase 5: Auto-Generation) - Addresses elastic#255820 (kbn/evals <-> Agent Builder completeness) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

…ation HIGH #5: Create workflow logger adapter - eliminates all 'as Logger' type casts - Adapts WorkflowLogger to Kibana Logger interface - Type-safe (no casts), supports debug/info/warn/error/fatal/trace - Used in all workflow steps (dedup, extract, tag) HIGH #7: Optimize case matching with inverted index - 99% performance gain - Created CaseEntityIndex class: entity → case_ids map - Before: O(n×m×k) = 500×100×20 = 1M comparisons - After: O(n×k) = 500×20 = ~11K ops - Scales to 10,000+ open cases (vs 100 limit before) - Logs candidate reduction stats MEDIUM elastic#10: Add entity validation - prevents malformed data - Created ENTITY_VALIDATORS for all 13 observable types - IPv4/IPv6 format validation, hash length checks, hostname RFC compliance - Filters invalid entities before case matching (improves quality) - Permissive for test fixtures (relaxed hash/agent_id rules) Progress: 8/12 deep review issues fixed Tests: 62/62 passing ✅ Types: 0 errors ✅

@timestamp

Closes elastic#258318 Closes elastic#258319 ## Summary Adds logic to the alert episodes table to display `.alert_actions` information. This includes: - New action-specific API paths. - Snooze - **Per group hash.** - Button in the actions column opens a popover where an `until` can be picked. - **When snoozed** - A bell shows up in the status column. - Mouse over the bell icon to see until when the snooze is in effect. - Unsnooze - **Per group hash.** - Clicking the button removes the snooze. - Ack/Unack - **Per episode.** - Button in the actions column - When "acked", an icon shows in the status column. - Tags - This PR only handles displaying tags. They need to be created via API. - Resolve/Unresolve - **Per group hash.** - Button inside the ellipsis always - The status is turned to `inactive` **regardless of the "real" status.** <img width="1704" height="672" alt="Screenshot 2026-03-25 at 16 04 12" src="https://github.com/user-attachments/assets/5ef4111a-6e0c-4114-a60e-ce5f81a86ac6" /> ## Testing <details> <summary>POST mock episodes</summary> ``` POST _bulk { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:00:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:01:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:02:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:03:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:04:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:05:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:06:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:07:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:08:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:09:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:10:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:11:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:12:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:13:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:19:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-4", "episode": { "id": "ep-004", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:22:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-5", "episode": { "id": "ep-005", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:26:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-9", "episode": { "id": "ep-006", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "gh-7", "episode": { "id": "ep-007", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "gh-7", "episode": { "id": "ep-007", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "gh-8", "episode": { "id": "ep-008", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "gh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "gh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "no_data" } ``` </details> - In the POST above, episodes 1 and 3, and episodes 6 and 9 have the same group hashes. - Go to `https://localhost:5601/app/observability/alerts-v2` and try all buttons. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

- reverts change to x-pack/plugins/elastic_assistant/server/lib/langc…

899b1c8

…hain/graphs/default_assistant_graph/prompts.ts

andrew-goldstein merged commit d68fd93 into fix/langgraph-esql Aug 2, 2024

andrew-goldstein deleted the fix/langgraph-esql-dry-run-2 branch August 2, 2024 21:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reverts change to x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/prompts.ts#7

Reverts change to x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/prompts.ts#7
andrew-goldstein merged 1 commit intofix/langgraph-esqlfrom
fix/langgraph-esql-dry-run-2

andrew-goldstein commented Aug 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

andrew-goldstein commented Aug 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant