Skip to content

docs: add business KPI best practice to metrics page - #3720

Merged
JackYPCOnline merged 1 commit into
strands-agents:mainfrom
azarboon:patch-3
Aug 10, 2026
Merged

JackYPCOnline merged 1 commit into
strands-agents:mainfrom
azarboon:patch-3

Conversation

@azarboon

@azarboon azarboon commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds a best practice to the metrics page recommending that teams track business-outcome KPIs (for example, resolution rate, escalation rate, task completion, and customer satisfaction) alongside SDK-generated technical metrics.

Currently, the metrics page primarily focuses on SDK-emitted metrics, which may leave readers with the impression that metrics are limited to what the SDK tracks. In contrast, the Observability page already describes a broader set of metrics, including customer feedback and retention data. As a result, the metrics page's best practices do not reflect that broader scope. This addition aligns with the "Operate by KPIs that map to business outcomes" design principle in the AWS Well-Architected Framework Agentic AI Lens, which is linked for further guidance.

Related Issues

Documentation PR

Type of Change

Documentation update

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@azarboon
azarboon requested a review from a team as a code owner August 8, 2026 09:40
@azarboon
azarboon requested a review from maisieyanz August 8, 2026 09:40
@github-actions github-actions Bot added the documentation Documentation changes, improvements, additions, content updates, site improvements, examples, guides label Aug 8, 2026
@JackYPCOnline

Copy link
Copy Markdown
Contributor

@strandly-the-agent is that correct? do we need to add it?

@strandly-the-agent

Copy link
Copy Markdown
Contributor

TL;DR: the content is correct — I checked it against the linked page, and it's close to verbatim. I'd take it, but with one tweak: as written it's advice the page gives the reader no way to act on.

Correct? Yes. The lens's Operational Excellence design principle "Operate by KPIs that map to business outcomes" says: "Resolution rate, escalation rate, customer satisfaction, and task completion sit alongside infrastructure metrics with equal weight in dashboards and reviews." So the four example KPIs and the "equal weight" framing are accurately sourced, and the URL resolves. Precedent for citing this lens is already on the page — best practice #2 links the same doc (metrics.mdx on main), and it's the only page that does.

Need it? It's a reasonable addition, but the gap is that nothing the SDK emits covers business outcomes, and this page never says where they'd come from — so the item reads as generic advice on an otherwise concrete reference page. One clause pointing at the mechanism fixes that.

Suggested wording for item 5
  1. Combine Technical Metrics with Business KPIs: The metrics on this page tell you how the agent ran, not whether it accomplished its purpose. Also collect KPIs that measure business outcomes — resolution rate, escalation rate, customer satisfaction, task completion — and track them alongside the technical metrics with equal weight. The SDK's built-in metrics don't include these, so record them yourself as custom span attributes or custom spans; see Custom Attribute Tracking in the Traces documentation. For more information, see the AWS Well-Architected Framework Agentic AI Lens Design Principles for Operational Excellence.

The anchor targets the ### Custom Attribute Tracking heading in traces.mdx (trace_attributes / custom spans), and ./traces.md matches the relative-link style already used earlier in metrics.mdx. Optional and lower confidence: since the list sits outside the <Tabs> blocks, I'd keep it free of language-specific type names (EventLoopMetrics vs AgentMetrics), which the wording above does.

One thing I did not flag: the added line is ~600 chars, over the 90-char limit in .agents/references/mdx-authoring.md. 45 lines of this file already exceed it, including the item-2 precedent, so it's not something to hold this PR on.

Review, not a verdict — azarboon's call on the wording and yours on whether the item belongs.

@JackYPCOnline

Copy link
Copy Markdown
Contributor

Thanks for the contribution

@JackYPCOnline
JackYPCOnline enabled auto-merge (squash) August 10, 2026 15:59
@JackYPCOnline
JackYPCOnline merged commit ae44fe9 into strands-agents:main Aug 10, 2026
19 of 21 checks passed

This branch had an error being deployed

1 failed deployment
manual-approval 095bfeff Deployed Aug 8, 2026 by azarboon via Build and Deploy Preview #1081
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation changes, improvements, additions, content updates, site improvements, examples, guides size/xs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants