Skip to content

test(scenarios): cover active view agent surface trajectory - #11610

Merged
lalalune merged 1 commit into
developfrom
test/11355-active-view-trajectory
Jul 2, 2026
Merged

test(scenarios): cover active view agent surface trajectory#11610
lalalune merged 1 commit into
developfrom
test/11355-active-view-trajectory

Conversation

@lalalune

@lalalune lalalune commented Jul 2, 2026

Copy link
Copy Markdown
Member

Refs #11355

Summary

  • Fix active-view awareness injection for ACTION_PLANNER calls that use messages payloads, not only prompt strings.
  • Add unit coverage for prompt and messages planner payloads.
  • Add deterministic strict LLM-proxy scenario coverage for active-view shell navigate -> element snapshot -> planner VIEWS interact by element id -> serverInteract domain effects.

Evidence

  • bun run --cwd packages/agent test -- src/runtime/conversation-compactor-runtime.test.ts passed after rebase: 1 file, 43 tests.
  • SCENARIO_USE_LLM_PROXY=1 SCENARIO_LLM_PROXY_STRICT=1 bun --conditions eliza-source --tsconfig-override ../../tsconfig.json src/cli.ts run test/scenarios --scenario deterministic-active-view-agent-surface --lane pr-deterministic --report ../../.github/issue-evidence/11355-active-view-agent-surface/report.json --report-dir ../../.github/issue-evidence/11355-active-view-agent-surface/viewer --run-dir ../../.github/issue-evidence/11355-active-view-agent-surface/run --export-native ../../.github/issue-evidence/11355-active-view-agent-surface/native.jsonl passed after rebase: 1 scenario, 0 failures.
  • Manual evidence review recorded in .github/issue-evidence/11355-active-view-agent-surface/README.md.
  • Committed artifacts: report JSON, native JSONL, run viewer, report viewer.

Evidence N/A

  • Live-model trajectory: N/A in this environment because no provider credentials were available. This PR adds the keyless strict LLM-proxy lane and documents that limitation in the evidence README.
  • UI screenshots/video: N/A, no app UI rendering changes; this is runtime prompt plumbing plus scenario-runner coverage.

Notes

  • The scenario runner command prints a Bun internal directory mismatch warning after completion, but exits 0 and writes passing artifacts.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 754f63b4-499e-49a5-a14c-fad988cf1a1d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/11355-active-view-trajectory

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lalalune
lalalune merged commit 053eb8c into develop Jul 2, 2026
32 of 57 checks passed
@lalalune
lalalune deleted the test/11355-active-view-trajectory branch July 2, 2026 22:45
lalalune added a commit that referenced this pull request Jul 3, 2026
…red the coverage classification gate (#11753)

deterministic-active-view-agent-surface (added by #11610) declares its
RESPONSE_HANDLER/ACTION_PLANNER fixtures via the harness
stage1ResponseHandlerFixture builder imported directly from
@elizaos/test-harness, which the strict-spec source scan didn't follow —
so the scenario fit neither classification bucket and the
'every deterministic message scenario is classified' gate has been red on
develop since that merge.

- register it in STRICT_LLM_ROUTING_SCENARIOS (VIEWS, 2 message turns)
- follow direct stage1ResponseHandlerFixture imports into the harness
  source for the fixture-literal checks, same as the
  registerStrictActionRouteFixtures re-export path

17/17 green (was 16/17).

Co-authored-by: Shaw <shawgotbags@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
lalalune pushed a commit that referenced this pull request Jul 3, 2026
…ecognize fixture-template imports in the coverage guard

#11610 added deterministic-active-view-agent-surface without classifying it
in the deterministic-action-coverage guard, leaving the suite red on develop.
Classify it as strict-routed (VIEWS, 2 message turns) and teach the guard's
fixture-literal check to read the canonical @elizaos/test-harness template
when a scenario builds fixtures from it directly (stage1ResponseHandlerFixture)
rather than only via registerStrictActionRouteFixtures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

LifeOps Benchmark — eliza

Run ID: lifeops-eliza-28623101268

LifeOps Benchmark

Model: gemma-4-31b
Judge: claude-opus-4-7
Scenarios: 25
pass@1: 0.000
pass@k: 0.000
Total cost: $0.0000

Full artifacts: see the lifeops-run-eliza-28623101268 upload on this run.

@claude

claude Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude encountered an error —— View job


I'll analyze this and get back to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant