closed — superseded by #1489 - #1488
Closed
ascerra wants to merge 10 commits into
Closed
Conversation
Instrument the fullsend CLI with structured telemetry (run-events.jsonl, run-summary.json) and optional OTLP export via OTEL_EXPORTER_OTLP_ENDPOINT. Implements W3C traceparent propagation for cross-run trace correlation. - internal/telemetry: new package with Recorder, InstrumentedPrinter, OTEL tracer init, and propagation utilities - internal/cli/run.go: wire InstrumentedPrinter into all lifecycle steps - ADR-0040: document the decision and consequences Relates-to: ADR-0040 Co-authored-by: Cursor <cursoragent@cursor.com>
The createOnce function shared a single context between the create command and the subsequent get/ready-poll checks. When openshell sandbox create hangs at "Requesting compute..." and consumes the full context timeout, the get check fails immediately (expired context) and the retry logic deletes the still-initializing sandbox. Use independent contexts for the existence check and ready poll so the full ready timeout is available even when the create command times out. Co-authored-by: Cursor <cursoragent@cursor.com>
When PHOENIX_URL secret is set in the caller repo, the reusable triage workflow builds the telemetry-enabled fullsend binary from the distributed-tracing branch and configures OTLP export to Phoenix Arize. The dispatch routes triage to the instrumented workflow via secrets: inherit so PHOENIX_URL and PHOENIX_INGEST_TOKEN flow through. Co-authored-by: Cursor <cursoragent@cursor.com>
GitHub Actions does not allow secrets context in step `if:` expressions. Removed the conditionals — the telemetry build always runs on this branch. Also reordered steps so the telemetry source is checked out before setup-go reads its go.mod. Co-authored-by: Cursor <cursoragent@cursor.com>
Make GCP and PHOENIX secrets optional in both reusable-dispatch and reusable-triage to work with explicit secret passing from caller. Co-authored-by: Cursor <cursoragent@cursor.com>
…spans in-process - SetRootStatus(OK/ERROR) called in defer block so root spans no longer show UNSET in Phoenix - SetRootAttribute adds fullsend.iterations, fullsend.exit_code, and fullsend.validation to root span - AddEvent/AddRootEvent methods on Recorder and InstrumentedPrinter for attaching OTEL span events - EmitTranscriptSpans parses claude.jsonl in-process and creates child spans under the agent-execution iteration span - Enriched step attributes on load-harness, pre-script, create-sandbox, validation, and other lifecycle steps Co-authored-by: Cursor <cursoragent@cursor.com>
The Go EmitTranscriptSpans call was creating 0ms-duration LLM spans that duplicated the richer ones from the Python transcript exporter. Remove it; the post-run Python script is the single source of truth. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace Arize Phoenix with MLflow as the OTLP trace ingestion backend. MLflow provides GenAI-aware dashboards with automatic token usage rollups and runs on a dedicated GCP VM at mlflow-35-212-57-52.nip.io. - Update reusable-triage.yml to use MLflow OTLP endpoint with bearer auth - Update reusable-dispatch.yml to pass MLFLOW_OTLP_TOKEN secret - Add OTEL_EXPORTER_OTLP_TRACES_ENDPOINT support in ConfigFromEnv and InitTracer (the traces-specific env var is what the OTEL SDK reads for per-signal configuration) - Add test for traces-specific endpoint auto-enabling telemetry - Document MLflow endpoint, auth, and infrastructure in admin guide Co-authored-by: Cursor <cursoragent@cursor.com>
Root span name now includes the work item ID (e.g. "triage: org/repo#74") instead of generic "triage-run", making traces identifiable in MLflow's trace list without clicking into each one. Co-authored-by: Cursor <cursoragent@cursor.com>
… guide updates Add approach evaluation matrix comparing framework-native vs post-hoc instrumentation across infrastructure requirements, timing accuracy, and GenAI dashboard support. Link to related ADRs (0018, 0021, 0022) and six open GitHub issues. Document transcript-to-span promotion, work_item_id correlation, InstrumentedPrinter buffering, and regression gate details. Move live deployment example from ADR to admin guide where it belongs as a living reference. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
Replaced by a clean PR with only the 2 doc files (1 commit). The original PR accidentally included all commits on the distributed-tracing branch. |
ascerra
changed the title
Contributor
Author
|
Closed because I didn't mean to push all this just trying to get the ADR and working guide out for sharing here #1489 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR was opened in error with the wrong branch scope. It has been superseded by #1489.
No content from this PR should be referenced.