Observability: OTel semantic conventions for agentic systems by david-martin · Pull Request #33 · kubernetes-sigs/kube-agentic-networking

david-martin · 2025-12-19T18:17:06Z

What type of PR is this?

/kind documentation

What this PR does / why we need it:

Defines tracing schemas for agentic systems by following OpenTelemetry semantic conventions, with proposed extensions for permission enforcement outcomes.

k8s-ci-robot · 2025-12-19T18:17:09Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

rubambiza

Thanks for kicking off the proposal @david-martin . I am a clean slate to the world of observability, so excuse any obvious questions in my review 😅

linux-foundation-easycla · 2026-01-08T10:01:34Z

The committers listed above are authorized under a signed CLA.

✅ login: david-martin / name: David Martin (34366d4, c7505fa, d60d9d3, db8bda8, 5aececc, 5d61225, bfc70f7, cc31050, d8d7572, fa884b8)
✅ login: evaline-ju / name: Evaline Ju (0d13644, 0ddaf61, 41c28bf, 80107f6, a583c4a, acc7621, c5fd604, dc70015, ef98f98)

david-martin · 2026-01-15T13:18:28Z

Capturing the outcome of conversation on slack & meet.

focus on schema first, with traces. API (including how to specify which fields to log/not log/redact) comes in a later proposal/addition, as will metrics
leaning into proxy use cases, let's iterate on an auditing example, building on something like an AccessPolicy rule that denies a tool call
Open q still if we want to add a schema repo. Do we have a handle on if/when https://github.com/open-telemetry/opentelemetry-specification/pull/4815/changes is going to land, and if that's a blocker for hosting our own schema?
- that PR indeed needs to land and there's still some debate ongoing regarding SemConv V2. Once we have the schema specified we can also contribute it to the existing SemConv. We could request help from this group: https://contribute.cncf.io/community/tcgs/artificial-intelligence/
  They've been driving the SemConv for agentic workloads.

netlify · 2026-01-27T13:23:06Z

✅ Deploy Preview for kube-agentic-networking ready!

Name	Link
🔨 Latest commit	`34366d4`
🔍 Latest deploy log	https://app.netlify.com/projects/kube-agentic-networking/deploys/699c30165ed0f900082fbca9
😎 Deploy Preview	https://deploy-preview-33--kube-agentic-networking.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Signed-off-by: David Martin <davmarti@redhat.com>

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

…n attributes from otel Signed-off-by: David Martin <davmarti@redhat.com>

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

📝 Consolidate span attributes

…for the most part Signed-off-by: David Martin <davmarti@redhat.com>

Simplify proposal to make it clear we're not defining new attributes …

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

📝 Add alternate conventions section

LiorLieberman · 2026-02-19T00:58:05Z

Overall this proposal looks good to me, thanks @david-martin!

/approve

/hold Will defer to @gkhom for final approval.

Signed-off-by: David Martin <davmarti@redhat.com>

aabmass · 2026-02-19T21:11:14Z

+### Guardrailing
+
+This shows a trace example of a guardrail blocking a request at a gateway.


👋 from OpenTelemetry GenAI SIG

I don't have a ton of context on this proposal, but I wanted to share a proposal for Guardrails open-telemetry/semantic-conventions#3233

aabmass · 2026-02-19T21:12:47Z

+      │
+      │ POST /mcp
+      │ Headers:
+      │   traceparent: 00-c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8-9c1d2e3f4a5b6c7d-01


FYI, propagation needs to happen in the JSON-RPC layer for MCP because of it's stateful nature.

SEP414 SEP-414: Document OpenTelemetry Trace Context Propagation Conventions modelcontextprotocol/modelcontextprotocol#414

OpenTelemetry semantic conventions https://github.com/open-telemetry/semantic-conventions/blob/v1.39.0/docs/gen-ai/mcp.md

rubambiza

Mostly clarifying questions, not sure if they may require any changes. They certainly do not need to block merging.

rubambiza · 2026-02-20T02:30:41Z

+      │     }
+      │   }
+      │ }
+      ▼


Is this the point where the traffic hits a k8s cluster?

I would say the request is inside the MCP aware router/gateway, or in the MCP Server at this point.
Either of those could be in a k8s cluster, yes.
After this point, it's up to the MCP server where that traffic goes (if anywhere), before the response is returned.

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

📝 Update traceparent and JSON-RPC bodies in examples

LiorLieberman · 2026-02-24T18:21:22Z

@david-martin anything else thats blocking you from getting this in?

david-martin · 2026-02-24T18:54:12Z

@LiorLieberman I think we're good to go here.

Thanks to everyone for the help and feedback on getting this to a mergable state.

LiorLieberman · 2026-02-24T19:01:20Z

/approve
/lgtm
@david-martin please remove the hold when ready

k8s-ci-robot · 2026-02-24T19:01:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: david-martin, LiorLieberman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [LiorLieberman,david-martin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

david-martin · 2026-02-24T19:05:07Z

/unhold

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/documentation Categorizes issue or PR as related to documentation. labels Dec 19, 2025

k8s-ci-robot requested a review from LiorLieberman December 19, 2025 18:17

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 19, 2025

gkhom reviewed Dec 19, 2025

View reviewed changes

Comment thread docs/proposals/0033-Observabilty.md Outdated

Comment thread docs/proposals/0033-Observabilty.md Outdated

Comment thread docs/proposals/0033-Observabilty.md Outdated

Comment thread docs/proposals/0033-Observabilty.md Outdated

rubambiza reviewed Jan 6, 2026

View reviewed changes

Comment thread docs/proposals/0033-Observabilty.md Outdated

Comment thread docs/proposals/0033-Observabilty.md Outdated

gkhom reviewed Jan 15, 2026

View reviewed changes

Comment thread docs/proposals/0033-Observabilty.md Outdated

evaline-ju mentioned this pull request Jan 20, 2026

📝 Update with security rule attributes david-martin/kube-agentic-networking#3

Merged

david-martin marked this pull request as ready for review January 22, 2026 16:40

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 22, 2026

haiyanmeng reviewed Jan 22, 2026

View reviewed changes

Comment thread docs/proposals/0033-Observability.md Outdated

howardjohn reviewed Jan 26, 2026

View reviewed changes

Comment thread docs/proposals/0033-Observability.md Outdated

Comment thread docs/proposals/0033-Observability.md Outdated

Comment thread docs/proposals/0033-Observability.md

david-martin and others added 6 commits January 27, 2026 14:20

Add Observability proposal

5aececc

Signed-off-by: David Martin <davmarti@redhat.com>

📝 Updates to observability proposal

0ddaf61

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

Add 3 possible solutions for propagation/correlation, and info on spa…

fa884b8

…n attributes from otel Signed-off-by: David Martin <davmarti@redhat.com>

📝 Add to proposal summary and trace examples

ef98f98

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

🚚 Rename title typo

a583c4a

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

📝 Update using security rule

acc7621

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

Merge pull request #5 from evaline-ju/obsv

c7505fa

📝 Consolidate span attributes

evaline-ju mentioned this pull request Feb 5, 2026

epic: Tracing observability PoC with K8s agentic networking kagenti/kagenti#619

Open

6 tasks

Simplify proposal to make it clear we're not defining new attributes …

bfc70f7

…for the most part Signed-off-by: David Martin <davmarti@redhat.com>

david-martin mentioned this pull request Feb 11, 2026

Simplify proposal to make it clear we're not defining new attributes … david-martin/kube-agentic-networking#6

Merged

evaline-ju mentioned this pull request Feb 11, 2026

REQUEST: New membership for evaline-ju kubernetes/org#6133

Closed

11 tasks

rubambiza mentioned this pull request Feb 11, 2026

REQUEST: New membership for @rubambiza kubernetes/org#6134

Closed

11 tasks

Merge pull request #6 from david-martin/simplify-obs-proposal

db8bda8

Simplify proposal to make it clear we're not defining new attributes …

david-martin changed the title ~~Add Observability auditing proposal~~ Observability: OTel semantic conventions for agentic systems Feb 12, 2026

evaline-ju and others added 2 commits February 13, 2026 09:39

📝 Add alternate conventions section

c5fd604

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

Merge pull request #7 from evaline-ju/obsv

d60d9d3

📝 Add alternate conventions section

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 19, 2026

david-martin added 2 commits February 19, 2026 12:28

Allow for multiple rule spans

d8d7572

Signed-off-by: David Martin <davmarti@redhat.com>

Include info and guardrails example for rules spans

cc31050

Signed-off-by: David Martin <davmarti@redhat.com>

david-martin force-pushed the observability-proposal branch from 7a3c85e to cc31050 Compare February 19, 2026 15:42

aabmass reviewed Feb 19, 2026

View reviewed changes

rubambiza reviewed Feb 20, 2026

View reviewed changes

evaline-ju and others added 3 commits February 20, 2026 12:34

🐛 Update traceparents in examples

dc70015

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

📝 Clarify JSON-RPC layer for MCP tool call examples

0d13644

Signed-off-by: Evaline Ju <69598118+evaline-ju@users.noreply.github.com>

Merge pull request #8 from evaline-ju/obsv

34366d4

📝 Update traceparent and JSON-RPC bodies in examples

k8s-ci-robot assigned LiorLieberman Feb 24, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 24, 2026

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 24, 2026

k8s-ci-robot merged commit f1b855d into kubernetes-sigs:main Feb 24, 2026
14 checks passed

evaline-ju mentioned this pull request Mar 12, 2026

Support emitting dynamic metadata for observability via the RBAC filter #170

Open

		### Guardrailing

		This shows a trace example of a guardrail blocking a request at a gateway.

Conversation

david-martin commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Dec 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rubambiza left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

linux-foundation-easycla Bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

david-martin commented Jan 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

netlify Bot commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kube-agentic-networking ready!

Uh oh!

LiorLieberman commented Feb 19, 2026

Uh oh!

aabmass Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

aabmass Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

rubambiza left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rubambiza Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

david-martin Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

LiorLieberman commented Feb 24, 2026

Uh oh!

david-martin commented Feb 24, 2026

Uh oh!

LiorLieberman commented Feb 24, 2026

Uh oh!

k8s-ci-robot commented Feb 24, 2026

Uh oh!

david-martin commented Feb 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

david-martin commented Dec 19, 2025 •

edited

Loading

linux-foundation-easycla Bot commented Jan 8, 2026 •

edited

Loading

netlify Bot commented Jan 27, 2026 •

edited

Loading