LLM Obs SDK use context API for parent children span linkage #8711

gary-huang · 2025-04-21T14:05:15Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-04-21T15:10:02Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/use-ctx-api
git_commit_date	1751991953	1751992518
git_commit_sha	`860a603`	`8c08304`
release_version	1.51.0-SNAPSHOT~860a603678	1.51.0-SNAPSHOT~8c083043c5

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1751994218	1751994218
ci_job_id	1018893332	1018893332
ci_pipeline_id	69940964	69940964
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-khiffj8e 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-khiffj8e 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (996.883 ms) : 0, 996883
Total [baseline] (8.569 s) : 0, 8568504
Agent [candidate] (1.001 s) : 0, 1001097
Total [candidate] (8.575 s) : 0, 8575239
section iast
Agent [baseline] (1.138 s) : 0, 1138103
Total [baseline] (9.264 s) : 0, 9264276
Agent [candidate] (1.136 s) : 0, 1136074
Total [candidate] (9.287 s) : 0, 9287121

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	996.883 ms	-
Agent	iast	1.138 s	141.22 ms (14.2%)
Total	tracing	8.569 s	-
Total	iast	9.264 s	695.771 ms (8.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.001 s	-
Agent	iast	1.136 s	134.977 ms (13.5%)
Total	tracing	8.575 s	-
Total	iast	9.287 s	711.881 ms (8.3%)

gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.555 ms) : 0, 688555
BytebuddyAgent [candidate] (691.485 ms) : 0, 691485
GlobalTracer [baseline] (242.175 ms) : 0, 242175
GlobalTracer [candidate] (243.244 ms) : 0, 243244
AppSec [baseline] (30.268 ms) : 0, 30268
AppSec [candidate] (30.475 ms) : 0, 30475
Debugger [baseline] (6.136 ms) : 0, 6136
Debugger [candidate] (6.083 ms) : 0, 6083
Remote Config [baseline] (694.866 µs) : 0, 695
Remote Config [candidate] (679.19 µs) : 0, 679
Telemetry [baseline] (8.241 ms) : 0, 8241
Telemetry [candidate] (8.261 ms) : 0, 8261
section iast
BytebuddyAgent [baseline] (813.041 ms) : 0, 813041
BytebuddyAgent [candidate] (810.633 ms) : 0, 810633
GlobalTracer [baseline] (232.211 ms) : 0, 232211
GlobalTracer [candidate] (232.518 ms) : 0, 232518
AppSec [baseline] (31.384 ms) : 0, 31384
AppSec [candidate] (30.531 ms) : 0, 30531
Debugger [baseline] (5.845 ms) : 0, 5845
Debugger [candidate] (5.829 ms) : 0, 5829
Remote Config [baseline] (571.942 µs) : 0, 572
Remote Config [candidate] (575.892 µs) : 0, 576
Telemetry [baseline] (7.977 ms) : 0, 7977
Telemetry [candidate] (8.015 ms) : 0, 8015
IAST [baseline] (26.224 ms) : 0, 26224
IAST [candidate] (27.121 ms) : 0, 27121

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.002 s) : 0, 1001943
Total [baseline] (10.775 s) : 0, 10775197
Agent [candidate] (997.054 ms) : 0, 997054
Total [candidate] (10.695 s) : 0, 10695473
section appsec
Agent [baseline] (1.201 s) : 0, 1200517
Total [baseline] (10.856 s) : 0, 10856386
Agent [candidate] (1.177 s) : 0, 1177127
Total [candidate] (10.742 s) : 0, 10742345
section iast
Agent [baseline] (1.134 s) : 0, 1134350
Total [baseline] (10.89 s) : 0, 10890412
Agent [candidate] (1.132 s) : 0, 1132019
Total [candidate] (10.861 s) : 0, 10861225
section profiling
Agent [baseline] (1.256 s) : 0, 1256236
Total [baseline] (10.954 s) : 0, 10954008
Agent [candidate] (1.246 s) : 0, 1246184
Total [candidate] (10.94 s) : 0, 10940263

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.002 s	-
Agent	appsec	1.201 s	198.574 ms (19.8%)
Agent	iast	1.134 s	132.407 ms (13.2%)
Agent	profiling	1.256 s	254.294 ms (25.4%)
Total	tracing	10.775 s	-
Total	appsec	10.856 s	81.189 ms (0.8%)
Total	iast	10.89 s	115.215 ms (1.1%)
Total	profiling	10.954 s	178.81 ms (1.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	997.054 ms	-
Agent	appsec	1.177 s	180.073 ms (18.1%)
Agent	iast	1.132 s	134.965 ms (13.5%)
Agent	profiling	1.246 s	249.13 ms (25.0%)
Total	tracing	10.695 s	-
Total	appsec	10.742 s	46.873 ms (0.4%)
Total	iast	10.861 s	165.753 ms (1.5%)
Total	profiling	10.94 s	244.79 ms (2.3%)

gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.209 ms) : 0, 691209
BytebuddyAgent [candidate] (688.524 ms) : 0, 688524
GlobalTracer [baseline] (243.383 ms) : 0, 243383
GlobalTracer [candidate] (242.312 ms) : 0, 242312
AppSec [baseline] (30.534 ms) : 0, 30534
AppSec [candidate] (30.368 ms) : 0, 30368
Debugger [baseline] (6.127 ms) : 0, 6127
Debugger [candidate] (6.058 ms) : 0, 6058
Remote Config [baseline] (692.311 µs) : 0, 692
Remote Config [candidate] (680.098 µs) : 0, 680
Telemetry [baseline] (9.046 ms) : 0, 9046
Telemetry [candidate] (8.222 ms) : 0, 8222
section appsec
BytebuddyAgent [baseline] (727.685 ms) : 0, 727685
BytebuddyAgent [candidate] (711.406 ms) : 0, 711406
GlobalTracer [baseline] (239.3 ms) : 0, 239300
GlobalTracer [candidate] (235.543 ms) : 0, 235543
AppSec [baseline] (173.664 ms) : 0, 173664
AppSec [candidate] (171.452 ms) : 0, 171452
Debugger [baseline] (5.861 ms) : 0, 5861
Debugger [candidate] (5.822 ms) : 0, 5822
Remote Config [baseline] (613.585 µs) : 0, 614
Remote Config [candidate] (612.353 µs) : 0, 612
Telemetry [baseline] (8.163 ms) : 0, 8163
Telemetry [candidate] (8.181 ms) : 0, 8181
IAST [baseline] (23.978 ms) : 0, 23978
IAST [candidate] (23.21 ms) : 0, 23210
section iast
BytebuddyAgent [baseline] (808.491 ms) : 0, 808491
BytebuddyAgent [candidate] (807.219 ms) : 0, 807219
GlobalTracer [baseline] (232.934 ms) : 0, 232934
GlobalTracer [candidate] (232.013 ms) : 0, 232013
AppSec [baseline] (29.777 ms) : 0, 29777
AppSec [candidate] (30.463 ms) : 0, 30463
Debugger [baseline] (5.873 ms) : 0, 5873
Debugger [candidate] (5.872 ms) : 0, 5872
Remote Config [baseline] (578.944 µs) : 0, 579
Remote Config [candidate] (586.537 µs) : 0, 587
Telemetry [baseline] (8.061 ms) : 0, 8061
Telemetry [candidate] (7.918 ms) : 0, 7918
IAST [baseline] (27.819 ms) : 0, 27819
IAST [candidate] (27.105 ms) : 0, 27105
section profiling
BytebuddyAgent [baseline] (685.973 ms) : 0, 685973
BytebuddyAgent [candidate] (678.096 ms) : 0, 678096
GlobalTracer [baseline] (363.618 ms) : 0, 363618
GlobalTracer [candidate] (362.133 ms) : 0, 362133
AppSec [baseline] (31.711 ms) : 0, 31711
AppSec [candidate] (33.12 ms) : 0, 33120
Debugger [baseline] (11.32 ms) : 0, 11320
Debugger [candidate] (9.92 ms) : 0, 9920
Remote Config [baseline] (661.694 µs) : 0, 662
Remote Config [candidate] (667.507 µs) : 0, 668
Telemetry [baseline] (9.532 ms) : 0, 9532
Telemetry [candidate] (8.823 ms) : 0, 8823
ProfilingAgent [baseline] (104.364 ms) : 0, 104364
ProfilingAgent [candidate] (104.789 ms) : 0, 104789
Profiling [baseline] (104.388 ms) : 0, 104388
Profiling [candidate] (104.813 ms) : 0, 104813

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/use-ctx-api
git_commit_date	1751991953	1751992518
git_commit_sha	`860a603`	`8c08304`
release_version	1.51.0-SNAPSHOT~860a603678	1.51.0-SNAPSHOT~8c083043c5

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1751993989	1751993989
ci_job_id	1018893335	1018893335
ci_pipeline_id	69940964	69940964
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-ry6qdr0g 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-ry6qdr0g 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 12 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.411 ms) : 36117, 36705
.   : milestone, 36411,
appsec (46.131 ms) : 45729, 46532
.   : milestone, 46131,
code_origins (44.659 ms) : 44274, 45043
.   : milestone, 44659,
iast (43.923 ms) : 43528, 44318
.   : milestone, 43923,
profiling (49.724 ms) : 49219, 50230
.   : milestone, 49724,
tracing (42.552 ms) : 42202, 42901
.   : milestone, 42552,
section candidate
no_agent (37.319 ms) : 37014, 37623
.   : milestone, 37319,
appsec (46.548 ms) : 46137, 46959
.   : milestone, 46548,
code_origins (44.976 ms) : 44595, 45357
.   : milestone, 44976,
iast (44.491 ms) : 44104, 44878
.   : milestone, 44491,
profiling (49.166 ms) : 48714, 49618
.   : milestone, 49166,
tracing (43.014 ms) : 42661, 43367
.   : milestone, 43014,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.411 ms [36.117 ms, 36.705 ms]	-
appsec	46.131 ms [45.729 ms, 46.532 ms]	9.72 ms (26.7%)
code_origins	44.659 ms [44.274 ms, 45.043 ms]	8.248 ms (22.7%)
iast	43.923 ms [43.528 ms, 44.318 ms]	7.513 ms (20.6%)
profiling	49.724 ms [49.219 ms, 50.23 ms]	13.314 ms (36.6%)
tracing	42.552 ms [42.202 ms, 42.901 ms]	6.141 ms (16.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.319 ms [37.014 ms, 37.623 ms]	-
appsec	46.548 ms [46.137 ms, 46.959 ms]	9.229 ms (24.7%)
code_origins	44.976 ms [44.595 ms, 45.357 ms]	7.657 ms (20.5%)
iast	44.491 ms [44.104 ms, 44.878 ms]	7.172 ms (19.2%)
profiling	49.166 ms [48.714 ms, 49.618 ms]	11.848 ms (31.7%)
tracing	43.014 ms [42.661 ms, 43.367 ms]	5.695 ms (15.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.47 ms) : 4418, 4522
.   : milestone, 4470,
iast (9.278 ms) : 9128, 9428
.   : milestone, 9278,
iast_FULL (14.133 ms) : 13859, 14407
.   : milestone, 14133,
iast_GLOBAL (10.046 ms) : 9872, 10220
.   : milestone, 10046,
profiling (8.784 ms) : 8650, 8918
.   : milestone, 8784,
tracing (7.426 ms) : 7307, 7545
.   : milestone, 7426,
section candidate
no_agent (4.36 ms) : 4307, 4412
.   : milestone, 4360,
iast (9.337 ms) : 9185, 9490
.   : milestone, 9337,
iast_FULL (13.852 ms) : 13578, 14126
.   : milestone, 13852,
iast_GLOBAL (10.371 ms) : 10190, 10551
.   : milestone, 10371,
profiling (9.028 ms) : 8886, 9171
.   : milestone, 9028,
tracing (7.195 ms) : 7092, 7298
.   : milestone, 7195,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.47 ms [4.418 ms, 4.522 ms]	-
iast	9.278 ms [9.128 ms, 9.428 ms]	4.808 ms (107.6%)
iast_FULL	14.133 ms [13.859 ms, 14.407 ms]	9.663 ms (216.2%)
iast_GLOBAL	10.046 ms [9.872 ms, 10.22 ms]	5.576 ms (124.8%)
profiling	8.784 ms [8.65 ms, 8.918 ms]	4.314 ms (96.5%)
tracing	7.426 ms [7.307 ms, 7.545 ms]	2.956 ms (66.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.36 ms [4.307 ms, 4.412 ms]	-
iast	9.337 ms [9.185 ms, 9.49 ms]	4.978 ms (114.2%)
iast_FULL	13.852 ms [13.578 ms, 14.126 ms]	9.492 ms (217.7%)
iast_GLOBAL	10.371 ms [10.19 ms, 10.551 ms]	6.011 ms (137.9%)
profiling	9.028 ms [8.886 ms, 9.171 ms]	4.669 ms (107.1%)
tracing	7.195 ms [7.092 ms, 7.298 ms]	2.836 ms (65.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/use-ctx-api
git_commit_date	1751991953	1751992518
git_commit_sha	`860a603`	`8c08304`
release_version	1.51.0-SNAPSHOT~860a603678	1.51.0-SNAPSHOT~8c083043c5

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1751994466	1751994466
ci_job_id	1018893337	1018893337
ci_pipeline_id	69940964	69940964
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ne75efq1 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ne75efq1 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.526 s) : 15526000, 15526000
.   : milestone, 15526000,
appsec (15.023 s) : 15023000, 15023000
.   : milestone, 15023000,
iast (18.456 s) : 18456000, 18456000
.   : milestone, 18456000,
iast_GLOBAL (17.698 s) : 17698000, 17698000
.   : milestone, 17698000,
profiling (15.738 s) : 15738000, 15738000
.   : milestone, 15738000,
tracing (15.005 s) : 15005000, 15005000
.   : milestone, 15005000,
section candidate
no_agent (14.998 s) : 14998000, 14998000
.   : milestone, 14998000,
appsec (14.884 s) : 14884000, 14884000
.   : milestone, 14884000,
iast (18.477 s) : 18477000, 18477000
.   : milestone, 18477000,
iast_GLOBAL (17.895 s) : 17895000, 17895000
.   : milestone, 17895000,
profiling (15.127 s) : 15127000, 15127000
.   : milestone, 15127000,
tracing (14.825 s) : 14825000, 14825000
.   : milestone, 14825000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.526 s [15.526 s, 15.526 s]	-
appsec	15.023 s [15.023 s, 15.023 s]	-503.0 ms (-3.2%)
iast	18.456 s [18.456 s, 18.456 s]	2.93 s (18.9%)
iast_GLOBAL	17.698 s [17.698 s, 17.698 s]	2.172 s (14.0%)
profiling	15.738 s [15.738 s, 15.738 s]	212.0 ms (1.4%)
tracing	15.005 s [15.005 s, 15.005 s]	-521.0 ms (-3.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.998 s [14.998 s, 14.998 s]	-
appsec	14.884 s [14.884 s, 14.884 s]	-114.0 ms (-0.8%)
iast	18.477 s [18.477 s, 18.477 s]	3.479 s (23.2%)
iast_GLOBAL	17.895 s [17.895 s, 17.895 s]	2.897 s (19.3%)
profiling	15.127 s [15.127 s, 15.127 s]	129.0 ms (0.9%)
tracing	14.825 s [14.825 s, 14.825 s]	-173.0 ms (-1.2%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~8c083043c5, baseline=1.51.0-SNAPSHOT~860a603678
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (2.394 ms) : 2345, 2444
.   : milestone, 2394,
iast (2.179 ms) : 2117, 2241
.   : milestone, 2179,
iast_GLOBAL (2.231 ms) : 2168, 2293
.   : milestone, 2231,
profiling (2.033 ms) : 1983, 2083
.   : milestone, 2033,
tracing (1.995 ms) : 1947, 2043
.   : milestone, 1995,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.4 ms) : 2351, 2450
.   : milestone, 2400,
iast (2.193 ms) : 2131, 2256
.   : milestone, 2193,
iast_GLOBAL (2.23 ms) : 2167, 2293
.   : milestone, 2230,
profiling (2.039 ms) : 1989, 2089
.   : milestone, 2039,
tracing (1.996 ms) : 1948, 2044
.   : milestone, 1996,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.47 ms [1.458 ms, 1.481 ms]	-
appsec	2.394 ms [2.345 ms, 2.444 ms]	924.748 µs (62.9%)
iast	2.179 ms [2.117 ms, 2.241 ms]	709.399 µs (48.3%)
iast_GLOBAL	2.231 ms [2.168 ms, 2.293 ms]	761.05 µs (51.8%)
profiling	2.033 ms [1.983 ms, 2.083 ms]	563.06 µs (38.3%)
tracing	1.995 ms [1.947 ms, 2.043 ms]	525.058 µs (35.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	2.4 ms [2.351 ms, 2.45 ms]	928.02 µs (63.0%)
iast	2.193 ms [2.131 ms, 2.256 ms]	720.939 µs (49.0%)
iast_GLOBAL	2.23 ms [2.167 ms, 2.293 ms]	757.614 µs (51.5%)
profiling	2.039 ms [1.989 ms, 2.089 ms]	566.363 µs (38.5%)
tracing	1.996 ms [1.948 ms, 2.044 ms]	523.649 µs (35.6%)

* add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * impl llmobs agent and llmobs apis * impl llmobs agent * working writer * add support for llm message and tool calls * impl llmobs agent and llmobs apis * use new ctx api to track parent span * cleaned up whitespace * resolve merge conflicts * remaining merge conflicts * fix bad method call * fixed llmobs intake creation if llmobs not enabled * removed print statements * ran spotless * added tests for llmobsspanmapper * fixed coverage for tags --------- Co-authored-by: Nayeem Kamal <[email protected]> Co-authored-by: Nayeem Kamal <[email protected]>

* add APIs for llm obs sdk (#8135) * add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners * implement LLM Obs SDK spans APIs (#8390) * add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * spotless * add APIs for llm obs sdk (#8135) * add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners * rename ID to Id according to java naming conventions * Undo change to integrations-core submodule * fix build gradle * rm empty line * fix test * LLM Obs SDK Mapper (#8372) * add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * impl llmobs agent and llmobs apis * impl llmobs agent * working writer * add support for llm message and tool calls * cleaned up whitespace * resolve merge conflicts * remaining merge conflicts * fix bad method call * fixed llmobs intake creation if llmobs not enabled * removed print statements * added tests for llmobsspanmapper * fixed coverage for tags --------- Co-authored-by: Nayeem Kamal <[email protected]> * updated to master submodule * LLM Obs SDK use context API for parent children span linkage (#8711) * add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * impl llmobs agent and llmobs apis * impl llmobs agent * working writer * add support for llm message and tool calls * impl llmobs agent and llmobs apis * use new ctx api to track parent span * cleaned up whitespace * resolve merge conflicts * remaining merge conflicts * fix bad method call * fixed llmobs intake creation if llmobs not enabled * removed print statements * ran spotless * added tests for llmobsspanmapper * fixed coverage for tags --------- Co-authored-by: Nayeem Kamal <[email protected]> Co-authored-by: Nayeem Kamal <[email protected]> * LLM Obs SDK evaluation metrics submission (#8688) * add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * impl llmobs agent and llmobs apis * impl llmobs agent * working writer * add support for llm message and tool calls * impl llmobs agent and llmobs apis * use new ctx api to track parent span * add api for evals * working impl supporting both agentless and agent * handle null tags and default to default ml app if null or empty string provided in the override * cleaned up whitespace * resolve merge conflicts * remaining merge conflicts * fix bad method call * fixed llmobs intake creation if llmobs not enabled * removed print statements * ran spotless * ran spotless * added tests for llmobsspanmapper * fixed coverage for tags --------- Co-authored-by: Nayeem Kamal <[email protected]> Co-authored-by: Nayeem Kamal <[email protected]> * fix CODEOWNERS --------- Co-authored-by: Nayeem Kamal <[email protected]> Co-authored-by: Nayeem Kamal <[email protected]>

gary-huang added 13 commits April 10, 2025 15:09

add APIs for llm obs

5134311

add llm message class to support llm spans

b5c7a0a

add llm message class to support llm spans

7f8f586

impl llmobs agent and llmobs apis

9206dbe

support llm messages with tool calls

f5efd26

handle default model name and provider

76740e9

rm unneeded file

5e66a2f

impl llmobs agent and llmobs apis

75290af

impl llmobs agent

7b2db7c

working writer

d76c94d

add support for llm message and tool calls

e0b4510

impl llmobs agent and llmobs apis

87cdaf1

use new ctx api to track parent span

f78b2c5

gary-huang changed the title ~~Gary/use ctx api~~ LLM Obs SDK use context API for parent children span linkage Jun 26, 2025

cleaned up whitespace

52d3502

nayeem-kamal requested a review from dougqh July 2, 2025 17:49

Nayeem Kamal and others added 7 commits July 2, 2025 14:39

resolve merge conflicts

f54c468

remaining merge conflicts

688d984

merge with feature branch

e62d569

fix bad method call

157e53e

fixed llmobs intake creation if llmobs not enabled

ade08ca

removed print statements

973c09f

Merge branch 'gary/add-llm-obs-writer' into gary/use-ctx-api

384141d

nayeem-kamal marked this pull request as ready for review July 3, 2025 20:09

nayeem-kamal self-requested a review July 3, 2025 20:10

Nayeem Kamal and others added 4 commits July 3, 2025 16:22

ran spotless

c74a642

added tests for llmobsspanmapper

1eb8db3

fixed coverage for tags

75d85a2

Merge branch 'gary/add-llm-obs-writer' into gary/use-ctx-api

ec3cf25

gary-huang requested review from a team as code owners July 8, 2025 16:22

gary-huang requested review from manuel-alvarez-alvarez, randomanderson and shatzi and removed request for a team July 8, 2025 16:22

gary-huang marked this pull request as draft July 8, 2025 16:23

gary-huang removed request for a team, Mariovido, dougqh, manuel-alvarez-alvarez, randomanderson and shatzi July 8, 2025 16:23

gary-huang force-pushed the gary/use-ctx-api branch 2 times, most recently from 06e220a to 7c16dd2 Compare July 8, 2025 16:28

Merge branch 'gary/llmobs-sdk-merge' into gary/use-ctx-api

8c08304

gary-huang marked this pull request as ready for review July 8, 2025 19:33

gary-huang merged commit f6ed413 into gary/llmobs-sdk-merge Jul 8, 2025
497 of 503 checks passed

gary-huang deleted the gary/use-ctx-api branch July 8, 2025 19:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

LLM Obs SDK use context API for parent children span linkage #8711

LLM Obs SDK use context API for parent children span linkage #8711

gary-huang commented Apr 21, 2025

Uh oh!

pr-commenter bot commented Apr 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LLM Obs SDK use context API for parent children span linkage #8711

LLM Obs SDK use context API for parent children span linkage #8711

Conversation

gary-huang commented Apr 21, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pr-commenter bot commented Apr 21, 2025 •

edited

Loading