Skip to content

docs(evidence): add #11504 stage1 code-reply proof - #11566

Merged
lalalune merged 1 commit into
developfrom
docs/11504-stage1-evidence
Jul 2, 2026
Merged

lalalune merged 1 commit into
developfrom
docs/11504-stage1-evidence

Conversation

@lalalune

@lalalune lalalune commented Jul 2, 2026

Copy link
Copy Markdown
Member

Closes #11504

Summary

The runtime fix for #11504 landed in #11555 (c532e3b565e): isUnusableStage1Reply no longer defers valid bare-code or structured replies just because they contain repeated-character runs. This PR adds the missing live evidence artifact and manual review notes required to close the issue.

Evidence Added

  • .github/issue-evidence/11504-stage1-code-replies/README.md
  • .github/issue-evidence/11504-stage1-code-replies/live-cerebras-current/93432706-b3b2-08ea-ab6a-ba55340a8848/tj-74b5c8fef7d6a9.json

Manual review of the trajectory confirms:

  • status: finished
  • single messageHandler stage
  • model gemma-4-31b
  • selected contexts:["simple"] with requiresTool:false
  • returned an unfenced Python code body as direct_reply instead of I'm not sure how to answer that.
  • no API key or authorization strings found in the trajectory artifact

Verification

  • bun run --cwd packages/core test src/__tests__/message-runtime-stage1.test.ts -> 1 passed, 75 passed
  • bunx @biomejs/biome check packages/core/src/services/message.ts packages/core/src/__tests__/message-runtime-stage1.test.ts .github/issue-evidence/11504-stage1-code-replies/README.md -> Checked 2 files ... No fixes applied.
  • bun run --cwd packages/core typecheck -> pass
  • live gemma-4-31b Cerebras runtime capture -> pass, artifact above
  • bun run install:light -> pass, artifact sync skipped with ELIZA_SKIP_ARTIFACT_SYNC=1
  • full bun install attempted, but postinstall artifact sync was still at 7.0 MiB / 971 MiB with 141m 59s remaining, so it was stopped before completion
  • bun run verify currently fails before typecheck/lint in audit:type-safety-ratchet on current repo baseline (as unknown as: 77 / 76, ?? 0 core/agent/app-core: 376 / 375). This PR changes only evidence artifacts, with no production-source diff.

UI / Media

N/A - core runtime evidence only; no user-facing UI changed.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3bd6ba39-3612-4915-8e07-d0dc27cc00e9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/11504-stage1-evidence

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@lalalune
lalalune force-pushed the docs/11504-stage1-evidence branch from 53fe4d9 to c79bf9c Compare July 2, 2026 20:38

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@lalalune

lalalune commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

Reviewed locally as evidence-only, but GitHub will not let this account approve its own PR. Rebased onto origin/develop and pushed c79bf9c. I validated the trajectory JSON parses and matches the README claims: status=finished, one messageHandler stage, model=gemma-4-31b, contexts=[simple], requiresTool=false, and the replyText is a Python function body rather than the fallback. I also checked the README issue/artifact links, ran git diff --check, and ran a narrow credential-pattern scan for sk-/sk-proj/Bearer/Authorization/x-api-key shaped values with no matches. Biome ignores this .github evidence README in the current config, so there was no meaningful Biome file to check here.

@lalalune
lalalune merged commit faa3af9 into develop Jul 2, 2026
33 of 36 checks passed
@lalalune
lalalune deleted the docs/11504-stage1-evidence branch July 2, 2026 20:52
@claude

claude Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude encountered an error —— View job


I'll analyze this and get back to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

eliza-runtime: isUnusableStage1Reply defers valid gemma-4-31b code replies (humaneval 0.40 vs 1.00)

1 participant