Skip to content

blog: three calls we'd have gotten wrong building our background agent - #239

Merged
krrish-berri-2 merged 22 commits into
mainfrom
litellm_blog-lap-agent
May 28, 2026
Merged

blog: three calls we'd have gotten wrong building our background agent#239
krrish-berri-2 merged 22 commits into
mainfrom
litellm_blog-lap-agent

Conversation

@ishaan-berri

@ishaan-berri ishaan-berri commented May 27, 2026

Copy link
Copy Markdown
Contributor

Summary

A blog post about what we learned shipping a background coding agent on LAP — anchored around three design calls and the Slack cold-start scar that motivated them:

  1. No BASH on the brain. Brain runs in a persistent shared harness pod; sandbox is per-session E2B accessed through a four-tool surface. Cites Anthropic's Building Effective Agents for the augmented-LLM frame. Includes inline SVG diagram.
  2. Self-report when stuck. `report_issue` tool stops the loop and posts a Linear comment instead of producing a half-done PR. PR-filing rate dropped after launch; merge bar stayed high.
  3. Harness, not agent framework. Long-running HTTP server (`POST /session`, `GET /event`, …) instead of a function call. Made `BerriAI/lite-harness` swappable across opencode and claude-agent-sdk.

Honest numbers: 21 merged PRs, 41 open, 138 closed-not-merged, 50+ filed/month on `BerriAI/litellm` over the last 3 weeks. Plus the `#help-litellm` Slack triage volume.

Includes a real Slack screenshot of the 2-minute cold-start failure mode that motivated the brain/sandbox split.

Files

  • `blog/lap_internal_agent_30pct/index.md` — the post
  • `static/img/lap_shin_slack_slow_start.png` — Slack cold-start screenshot
  • `static/img/lap_brain_sandbox_split.svg` — inline architecture diagram

Test plan

  • Renders locally via `yarn start` — 200 on /blog/lap-internal-agent-30-percent
  • All four merged-PR links resolve
  • Both images load
  • Review tone before publishing

@vercel

vercel Bot commented May 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment May 28, 2026 4:12am

Request Review

@ishaan-berri ishaan-berri changed the title blog: how a background agent on LAP is closing 30% of merged tickets blog: three calls we'd have gotten wrong building our background agent May 28, 2026
…inks/typos

- Move "What we shipped" above the numbered learnings so the 30% proof lands early
- Tie the ~30% claim to weekly ticket throughput; reframe 138 closed-without-merge as the cheap-session design, not noise
- Convert numbered sections to H2 headings for sidebar nav
- Add {/* truncate */} fold marker
- Fix broken author-PR link and "its PRs"/"its memory"/"smart enough" typos

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Lead with the hook (21 merged, no human code + the MITM story) instead of a meta intro
- Tie the ~30% claim to eng tickets/week and restore the "138 closed by design" framing so the merge ratio reads as intent, not noise
- Embed the two staged assets: Slack cold-start screenshot and the brain/sandbox split diagram (the diagram also documents the no-BASH design the description promised)
- Add real artifacts: lite-harness repo contract and the credential-to-host vault config that kills the MITM
- End each section on a claim instead of a hedge; soften the demo CTA

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Replace hook with user's framing: "goal to 10x productivity" → "30% of tickets"
- Add required sections: Key Takeaways, FAQ (w/ OSS vs Enterprise), Conclusion, Recommended Reading
- Add ## Where the AI Gateway fits (required H2 + brings AI Gateway to 5 mentions)
- Update frontmatter: description w/ CTA hint, ai-gateway tag, hide_table_of_contents: true
- Remove rogue-agent lede (trust risk); MITM stays as §3 lesson

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove sandbox_read_file and sandbox_write_file; keep only
sandbox_provision and sandbox_execute. Reposition boxes centered,
update arrows, label, and matching prose.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
krrish-berri and others added 2 commits May 27, 2026 20:30
…ence)

Shows Ishaan's real-time Slack messages catching the agent circumventing
the credential stub vault and storing real keys in its memory.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
krrish-berri and others added 2 commits May 27, 2026 20:51
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
krrish-berri and others added 2 commits May 27, 2026 20:55
- Fix ambiguous OOM attribution in §2; fix "it's"→"its", "grew slower"→"grew more slowly"
- Tighten "unifies X to Y" → "adapts X to a single HTTP contract"
- Replace "big win in terms of" with concrete sentence
- Fix dangling -[blog] link; clean up admonition block links
- Fix §3 closing: "Our learning here was that" → "The lesson:"
- §4 (AI Gateway): remove filler opener, fix space-before-comma, fix "Furthermore"
- §5: "is the part that is mostly solved" → "is largely solved"; "do destructive actions" → fix; "MCP's" → "MCPs"; "more preferable" → "more reliable"
- Key Takeaway: fix contradiction with §3 on LLM-level guardrails
- FAQ: deduplicate Cursor answer; fix "self-hosted" as license attribute
- Try it: consolidate duplicate links from info block into one clean sentence
- Fix stale SVG alt text ("four tool calls" → "two tool calls")

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
krrish-berri and others added 3 commits May 27, 2026 20:58
Colons for appositives, commas for conjunctions, parens for
asides, periods for sentence breaks. Zero em dashes remain.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
krrish-berri and others added 3 commits May 27, 2026 21:09
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@krrish-berri-2
krrish-berri-2 merged commit 2a0e702 into main May 28, 2026
2 checks passed
@krrish-berri-2
krrish-berri-2 deleted the litellm_blog-lap-agent branch May 28, 2026 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants