Repository navigation
docs: agent operations roadmap + telemetry plan #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
e936e8b
docs: add agent operations roadmap and telemetry plan
nutt-adam ad6f468
feat(automation): add Codex SDLC orchestration framework and helper s…
nutt-adam 501d8e4
ci: add workflow_dispatch orchestrator for Tutti SDLC smoke/auto runs
nutt-adam a395cf7
docs: specialize Codex SDLC topology into focused independent agents
nutt-adam c48b1ac
docs(workflow): add Claude planner lane and no-commit guard to SDLC c…
nutt-adam 935237b
fix(workflow): remove missing plan-file dependency from SDLC cycle
nutt-adam c3dc86f
fix: always collect CodeRabbit feedback and claim selected issues
nutt-adam 916ab11
chore: address PR review actionables for roadmap and workflow
nutt-adam 4090d6a
fix: harden sdlc orchestration flow for reruns and worktree alignment
nutt-adam 59eec6f
fix: remove duplicate issue preselection and wire issue_label through…
nutt-adam 4465a31
refactor: extract SDLC inline python blocks into automation scripts
nutt-adam 6ff9e88
fix: harden coderabbit wait/collect script error handling
nutt-adam c53d7db
fix(automation): include inline CodeRabbit comments and strict succes…
wren-nutthouse 1fa8fd3
Merge branch 'main' into roadmap/agent-ops-telemetry
wren-nutthouse File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # Tutti Agent Operations Roadmap (Execution Plan) | ||
|
|
||
| ## Goal | ||
| Position Tutti as the agent operations layer (not just a coding assistant) by closing reliability, governance, observability, and adoption gaps. | ||
|
|
||
| ## Priorities | ||
|
|
||
| ### P0 — Foundation (must ship first) | ||
| 1. **Identity + Auth + RBAC** | ||
| - User/org/workspace identities | ||
| - API keys and role-based permissions | ||
| - Audit trail for sensitive actions | ||
|
|
||
| 2. **Production Control Plane Hardening** | ||
| - Stable API contracts | ||
| - Durable event stream semantics | ||
| - Idempotency for mutating endpoints | ||
| - Queryable run/event history | ||
|
|
||
| 3. **Reliability Primitives** | ||
| - Retry policies + backoff | ||
| - Checkpoint/resume guarantees | ||
| - Dead-letter handling for failed actions | ||
| - Circuit breakers / provider failover | ||
|
|
||
| 4. **Approval and Guardrail Workflows** | ||
| - Human approval gates for risky actions | ||
| - Policy-as-code enforcement | ||
| - Exception handling with traceability | ||
|
|
||
| 5. **Operator Observability** | ||
| - Per-run timeline and failure stages | ||
| - Root cause attribution (routing/tool/model/policy) | ||
| - SLO dashboards and alerting hooks | ||
|
|
||
| ### P1 — Scale and Governance | ||
| 6. **Cost Governance** | ||
| - Workspace/agent budgets | ||
| - Forecasting and anomaly detection | ||
| - Hard-stop + escalation behavior | ||
|
|
||
| 7. **Deployment and Operations Story** | ||
| - Managed and self-hosted deployment paths | ||
| - Secrets/backup/migration tooling | ||
| - Upgrade safety and rollback docs | ||
|
|
||
| 8. **Ecosystem + Templates** | ||
| - Connector starter set (CRM, ticketing, docs, messaging) | ||
| - Template arrangements/workflows by use case | ||
| - <30 min time-to-first-value path | ||
|
|
||
| 9. **Trust and Compliance Pack** | ||
| - Security posture docs | ||
| - Data retention/export controls | ||
| - Auditability and incident response basics | ||
|
|
||
| 10. **Positioning + Onboarding Narrative** | ||
| - Replace generic “agent framework” messaging | ||
| - “Run reliable agent operations in one day” onboarding flow | ||
| - Product-led proof of value in first session | ||
|
|
||
| ## Telemetry Plan (parallel to all milestones) | ||
|
|
||
| ### North Star | ||
| - **Activation rate:** installs that reach first successful workflow run within 24h | ||
|
|
||
| ### Event instrumentation | ||
| - Install: `tutti_install_started`, `tutti_install_completed` | ||
| - Activation: `first_workspace_created`, `first_up_success`, `first_send`, `first_workflow_run` | ||
| - Habit: DAW (daily active workspaces), returning workspace cohorts (D1/D7/D30), runs per workspace | ||
| - Outcome: run success/failure, failure type (`routing/tool/model/policy`), time-to-success | ||
| - Commercial: conversion funnel install → first up → first successful run → D7 return | ||
|
|
||
| ### Telemetry deliverables | ||
| - Event schema + versioning | ||
| - Redaction/privacy defaults | ||
| - Dashboard for activation, reliability, retention | ||
| - Alerting on activation regression and failure spikes | ||
|
|
||
| ## Suggested sequencing (8-week sketch) | ||
| - **Weeks 1-2:** P0.1 + P0.2 + telemetry schema foundation | ||
| - **Weeks 3-4:** P0.3 + P0.4 + failure attribution dashboards | ||
| - **Weeks 5-6:** P0.5 + P1.6 (observability + cost governance) | ||
| - **Weeks 7-8:** P1.7 + P1.8 + onboarding narrative improvements | ||
| - Ongoing: P1.9 + P1.10 hardening and GTM tuning | ||
|
|
||
| ## Success criteria | ||
| - Activation >= target baseline and trending up | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| - Reduced run failure rate and faster MTTR | ||
| - Clear operator trust: approval/audit/cost controls usable in production | ||
| - Repeatable adoption via templates + onboarding story | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.