Initial distributed tracing instrumentation by sallyom · Pull Request #48 · llm-d/llm-d-kv-cache

sallyom · 2025-07-09T01:30:24Z

Initial distributed tracing instrumentation

liu-cong · 2026-02-14T00:43:17Z

 func (k *Indexer) GetPodScores(ctx context.Context, renderReq *preprocessing.ApplyChatTemplateRequest, prompt, modelName string,
 	podIdentifiers []string,
 ) (map[string]float64, error) {
+	// Start tracing span for main operation


+1 to get tracing encapsulated. Having some sparse inline tracing code is fine, but the bulk should be encapsulated. I would argue for keeping the inline bits, and wrap the indexer and scorer with tracing either in this PR or in a follow up. It's much cleaner that way without needing another refacor

github-actions · 2026-02-23T17:42:51Z

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

- Use global otel.Tracer() in library code for scheduler integration - Add telemetry initialization for standalone examples - Add gRPC interceptors for trace context propagation Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com>

Signed-off-by: sallyom <somalley@redhat.com>

Signed-off-by: greg pereira <grpereir@redhat.com>

…ationName Pass ctx through the Score interface so traced scorer spans are children of the parent get_scores span instead of orphaned traces. Export a single telemetry.InstrumentationName constant used by all tracing wrappers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com>

Signed-off-by: sallyom <somalley@redhat.com>

liu-cong

/lgtm

/approve

I left a couple of nits for the naming

Signed-off-by: greg pereira <grpereir@redhat.com>

vMaroon · 2026-02-25T15:50:41Z

/lgtm
/approve

* Add OpenTelemetry manual instrumentation for distributed tracing - Use global otel.Tracer() in library code for scheduler integration - Add telemetry initialization for standalone examples - Add gRPC interceptors for trace context propagation Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com> * fix: resolve lint errors in tracing instrumentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com> * fix: use url.Parse in stripScheme for robustness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com> * tracing: use block-level hit ratio instead of pod-level Signed-off-by: sallyom <somalley@redhat.com> * encapsulate the tracing logic Signed-off-by: greg pereira <grpereir@redhat.com> * Add context.Context to KVBlockScorer.Score and consolidate InstrumentationName Pass ctx through the Score interface so traced scorer spans are children of the parent get_scores span instead of orphaned traces. Export a single telemetry.InstrumentationName constant used by all tracing wrappers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com> * fix lint errors Signed-off-by: sallyom <somalley@redhat.com> * cong feedback about span naming conventions Signed-off-by: greg pereira <grpereir@redhat.com> --------- Signed-off-by: sallyom <somalley@redhat.com> Signed-off-by: greg pereira <grpereir@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: greg pereira <grpereir@redhat.com>

fix: nodeport service overwritten

sallyom mentioned this pull request Jul 9, 2025

distributed tracing proposal llm-d/llm-d#119

Merged

sallyom force-pushed the tracing branch 2 times, most recently from 3342ece to 74dac2d Compare July 30, 2025 00:53

frzifus reviewed Jul 31, 2025

View reviewed changes

Comment thread pkg/kvcache/indexer.go Outdated

Comment thread pkg/kvcache/indexer.go Outdated

sallyom force-pushed the tracing branch from 74dac2d to dfd910d Compare August 13, 2025 23:22

sallyom force-pushed the tracing branch 11 times, most recently from 41fd565 to 216b818 Compare August 27, 2025 17:49

sallyom force-pushed the tracing branch from 216b818 to 39b36ba Compare September 25, 2025 22:28

sallyom force-pushed the tracing branch 2 times, most recently from 8259996 to 19df391 Compare October 28, 2025 14:15

sallyom force-pushed the tracing branch 3 times, most recently from e289ca2 to 89b9fc9 Compare December 5, 2025 00:28

sallyom force-pushed the tracing branch from 89b9fc9 to 512d273 Compare January 5, 2026 16:39

vMaroon requested review from hyeongyun0916, liu-cong and yankay January 5, 2026 16:39

sallyom force-pushed the tracing branch from 512d273 to 4b314e8 Compare January 6, 2026 19:21

sallyom force-pushed the tracing branch from 4b314e8 to ee4142c Compare January 14, 2026 04:39

vMaroon requested a review from sagearc January 14, 2026 04:39

sallyom force-pushed the tracing branch from ee4142c to e5d247e Compare January 21, 2026 01:51

sallyom force-pushed the tracing branch from ad02a8a to c40edc0 Compare February 12, 2026 05:21

sallyom mentioned this pull request Feb 12, 2026

Distributed tracing implementation docs and guides llm-d/llm-d#745

Merged

4 tasks

liu-cong reviewed Feb 14, 2026

View reviewed changes

sallyom force-pushed the tracing branch from f6aa22c to 729bee1 Compare February 23, 2026 17:42

vMaroon requested a review from liu-cong February 23, 2026 17:43

sallyom force-pushed the tracing branch from 729bee1 to b635afd Compare February 23, 2026 21:36

liu-cong reviewed Feb 24, 2026

View reviewed changes

Comment thread pkg/kvcache/indexer.go Outdated

Gregory-Pereira mentioned this pull request Feb 24, 2026

encapsulate the tracing logic sallyom/llm-d-kv-cache#1

Merged

vMaroon requested a review from liu-cong February 24, 2026 22:12

sallyom force-pushed the tracing branch from 2b9af65 to 69b768d Compare February 24, 2026 22:55

sallyom and others added 7 commits February 24, 2026 19:36

fix: resolve lint errors in tracing instrumentation

a37672c

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com>

fix: use url.Parse in stripScheme for robustness

6d45e03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: sallyom <somalley@redhat.com>

tracing: use block-level hit ratio instead of pod-level

f08ca71

Signed-off-by: sallyom <somalley@redhat.com>

encapsulate the tracing logic

5da90cb

Signed-off-by: greg pereira <grpereir@redhat.com>

fix lint errors

fd6e8f9

Signed-off-by: sallyom <somalley@redhat.com>

sallyom force-pushed the tracing branch from 1468ed1 to fd6e8f9 Compare February 25, 2026 01:08

liu-cong previously approved these changes Feb 25, 2026

View reviewed changes

Comment thread pkg/kvcache/kvblock/traced_index.go Outdated

Comment thread pkg/kvcache/kvblock/traced_index.go Outdated

cong feedback about span naming conventions

99db36b

Signed-off-by: greg pereira <grpereir@redhat.com>

Gregory-Pereira dismissed liu-cong’s stale review via 99db36b February 25, 2026 15:13

vMaroon requested a review from liu-cong February 25, 2026 15:13

github-actions Bot added the lgtm Looks good to me, indicates that a PR is ready to be merged. label Feb 25, 2026

github-actions Bot approved these changes Feb 25, 2026

View reviewed changes

vMaroon merged commit 4130fbb into llm-d:main Feb 25, 2026
12 of 15 checks passed

guygir pushed a commit to guygir/llm-d-kv-cache-manager that referenced this pull request Apr 20, 2026

Merge pull request llm-d#48 from shaneutt/fix-nodeport

634133f

fix: nodeport service overwritten

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial distributed tracing instrumentation#48

Initial distributed tracing instrumentation#48
vMaroon merged 8 commits into
llm-d:mainfrom
sallyom:tracing

sallyom commented Jul 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

liu-cong Feb 14, 2026

Uh oh!

github-actions Bot commented Feb 23, 2026

Uh oh!

Uh oh!

liu-cong left a comment

Uh oh!

Uh oh!

Uh oh!

vMaroon commented Feb 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

sallyom commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

liu-cong Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Feb 23, 2026

Uh oh!

Uh oh!

liu-cong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vMaroon commented Feb 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sallyom commented Jul 9, 2025 •

edited

Loading