[APM] Add Service map PoC tests#23
Merged
Merged
Conversation
MiriamAparicio
approved these changes
Jan 20, 2026
MiriamAparicio
left a comment
MiriamAparicio
left a comment
There was a problem hiding this comment.
This is so much better! Good job
jennypavlova
pushed a commit
that referenced
this pull request
Apr 29, 2026
## Summary Closes elastic#254714 - Adds Claude reviewer using [GH Agentic Workflows](https://github.github.com/gh-aw/introduction/how-they-work/) - Adds workflow handling of @/claude mentions in comments - Adds a `code-reviewer` subagent for reuse in workflows. - Proxies through LiteLLM for Opus 4.7 model ### Testing https://github.com/elastic/kibana/actions/workflows/reviewer-claude.lock.yml This branch is directly in `elastic/kibana` rather than my fork and has been running the workflow for reviews during development with `pull_request` as the event, rather than the final state of `pull_request_target` events. You can see comments Claude has posted and responses to being mentioned. "No issues found" comments have been moved to a `no-op` instead, the status checks reflect when the agent runs anyways. Will need to follow up with a proper `pull_request_target` run after merge. There are some security settings automatically applied during compilation that we cannot verify yet. Since we're gated behind the label, there shouldn't be any issues immediately post merge with the workflow running unnecessarily. ### Note for Reviewers - It isn't necessary to review the two `lock` files. They are compiled and generated files, but must be committed for the workflow to run. - Open to different model versions, don't have a strong preference towards Opus 4.7 ## Cache and cost analysis Tested ten Opus 4.7 (`llm-gateway/claude-opus-4-7`) reviewer runs with `ENABLE_PROMPT_CACHING_1H` enabled. The cache is active, but the data includes dirty cases: prompt changes, long gaps outside the useful cache window, regular PR review runs, and comment-reply runs. | Run | Event | Head | Cost | Turns | Cache read | Cache write | Output | Effective tokens | | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | [#15](https://github.com/elastic/kibana/actions/runs/25011054153) | `pull_request` | `2076d4f` | $1.04 | 14 | 809,672 | 80,897 | 5,169 | 182,559 | | [#16](https://github.com/elastic/kibana/actions/runs/25011557405) | `pull_request` | `382357b` | $1.32 | 21 | 1,495,919 | 70,406 | 5,226 | 240,928 | | [#17](https://github.com/elastic/kibana/actions/runs/25012064350) | `pull_request` | `e93fe0b` | $1.33 | 23 | 1,378,836 | 50,413 | 13,022 | 240,413 | | [#18](https://github.com/elastic/kibana/actions/runs/25013588252) | `pull_request` | `04ded1d` | $1.20 | 24 | 1,453,092 | 39,648 | 8,876 | 220,490 | | [#19](https://github.com/elastic/kibana/actions/runs/25030187214) | `pull_request` | `88a2c7e` | $1.41 | 16 | 1,165,558 | 99,949 | 8,171 | 249,210 | | [#21](https://github.com/elastic/kibana/actions/runs/25030484848) | `pr_review_comment` | `88a2c7e` | $0.45 | 9 | 512,383 | 19,390 | 3,035 | 82,782 | | [#23](https://github.com/elastic/kibana/actions/runs/25031202052) | `pull_request` | `66b198f` | $1.39 | 18 | 1,230,107 | 78,104 | 11,420 | 246,818 | | [elastic#26](https://github.com/elastic/kibana/actions/runs/25033088024) | `pull_request` | `d59d8c0` | $1.50 | 22 | 1,428,302 | 90,868 | 8,885 | 269,265 | | [elastic#27](https://github.com/elastic/kibana/actions/runs/25034181833) | `pull_request` | `d1052ef` | $1.73 | 24 | 2,016,691 | 81,113 | 8,737 | 317,759 | | [elastic#28](https://github.com/elastic/kibana/actions/runs/25034226594) | `pr_review_comment` | `d1052ef` | $1.18 | 23 | 1,453,250 | 36,528 | 7,788 | 213,038 | | Run | First request cache read | First request cache write | Notes | | --- | ---: | ---: | --- | | 15 | 0 | 51,751 | Cold 1-hour cache write. | | 16 | 35,886 | 15,905 | Stable prefix reused. | | 17 | 35,886 | 15,907 | Same prefix hit pattern. | | 18 | 35,886 | 15,911 | Same prefix hit pattern. | | 19 | 0 | 53,676 | Cold again after long gap + prompt/head changes. | | 21 | 45,471 | 8,375 | Comment-reply flow reused 19 prefix and wrote a smaller suffix. | | 23 | 37,245 | 16,587 | New commit/prompt still reused part of the 1-hour prefix. | | 26 | 0 | 53,761 | Cold again after another gap + prompt/head changes. | | 27 | 37,245 | 16,511 | Warm PR run on a new head/prompt hash. | | 28 | 45,465 | 8,461 | Comment-reply flow reused 27 prefix, but still ran long. | Summary: - All observed cache creation was 1-hour (`ephemeral_1h_input_tokens`); no 5-minute writes were observed. - Average observed cost is about $1.26/run across all ten runs, or about $1.37 for normal `pull_request` runs excluding comment replies. - Comment replies are not inherently cheap: run 21 was $0.45 because it was narrow and 9 turns, while run 28 was $1.18 because it took 23 turns. - Cold starts appear after long gaps or prompt churn (runs 15, 19, 26); warm runs still reuse partial prefixes even when the commit/prompt hash changes. - Next tuning is comparing 5-minute vs 1-hour caching, testing cheaper models against review quality, and auditing trace data for unnecessary output or artifact reads. --------- Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds tests to the React Flow service map PoC and validates the functionality and the components rendering.
Coverage Comparison: React Flow vs Cytoscape
What React Flow Tests Add
Results
✅ All 89 component tests passing
✅ E2E tests verify real browser rendering
✅ Covers edge cases Cytoscape tests missed (health status, bidirectional edges, empty states)
Component Tests (Jest + React Testing Library)
Located in
x-pack/solutions/observability/plugins/apm/public/components/app/service_map/react_flow_service_map/react_flow_graph.test.tsxservice_node.test.tsxdependency_node.test.tsxreact_flow_popover.test.tsxtransform_data.test.tsapply_layout.test.tsTotal: 89 tests
Test Utilities
test_utils.tsx- Mock data factories, test wrapper with providers, predefined scenariosE2E Tests (Scout/Playwright)
Located in
x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel_tests/sevice_map/react_flow_service_map.spec.tsTests the
/app/apm/react-flow-service-maproute with real browser rendering.How to run
node scripts/scout start-server --statefulnpx playwright test --config=x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts --project=local --grep="React Flow Service Map"--ui