Structured Eval Results + OTel Logging#43398

Merged

needuv merged 68 commits intomainfrom

needuv/structured-results-otel-logging

Oct 22, 2025

Commits on Sep 30, 2025

skeleton code

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 1, 2025

add function that logs eval results to app insights

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
format

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
log red team data

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 8, 2025

get agent info from app insights config

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
merge with main

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 9, 2025

add eval result converter (#43233)
YoYoJa
authored
rename function

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix a thing

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 10, 2025

Jessli/convert (#43342)

authored
Fix usage (#43355)

authored
save

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'needuv/structured-results-otel-logging' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
add _type to evals/aoai graders

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 12, 2025

Jessli/convert make eval_meta_data optional (#43376)

authored

Commits on Oct 13, 2025

add error logging for otel event emission

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'needuv/structured-results-otel-logging' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix merge conflicts

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
add input/output tokens for prompty evals

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 14, 2025

Jessli/convert - update param name to add underscore (#43411)
YoYoJa
authored
exclude token counts from aggregation

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'needuv/structured-results-otel-logging' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
add total token count to prompty output

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix prompty tests

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 15, 2025

remove fields from app insights config

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
make new evaluation result fields private, and add a toggle in evaluate

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
change output fields to be private

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 16, 2025

Jessli/convert parse annotation and add trace_id (#43463)
YoYoJa
authored
Jessli/convert add trace_id, response_id, conversation_id (#43469)
YoYoJa
authored
refactor app insights push to prevent warnings

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
run black on code

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix merge conflicts

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
move otel import to internal module

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Jessli/convert expose sample data for sdk promty based evaluators (#43474)
YoYoJa
authored

Commits on Oct 17, 2025

Jessli/convert remove token counts from metrics (#43477)
YoYoJa
authored
Jessli/convert remove useless lines and fix UT (#43480)
YoYoJa
authored
try changing prompty output to dict

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'needuv/structured-results-otel-logging' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
change prompty output to dict

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
run black

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix relevance and prompty test

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix unit tests

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix prompty tests

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix similarity test

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
move groundedness to actual prompty impl

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
chore: Update assets.json
kdestin
committed

Commits on Oct 20, 2025

Jessli/convert Fix test failure (#43518)
YoYoJa
authored
add extra attributes to app insights config, remove agent name/id/version/ from app insights config

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
pin otel<1.39.0 since breaking change coming in that version

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
merge

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
implement scrubber for sensitive information

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
run black formatter

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix spelling for evaluation sdk

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
use non-deprecated path for emitting traces

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
remove upper bound on otel sdk

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
shuffle imports

Neehar Duvvuri
authored and
Neehar Duvvuri
committed

Commits on Oct 21, 2025

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Jessli/convert1021 Fxi bug (#43563)

authored
Jessli/convert1021 fix null value and black run (#43567)

authored
add span id logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
Merge branch 'needuv/structured-results-otel-logging' of https://github.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
[fix] enrich log attributes (#43522)

Zyysurely
and
yingyingzhaozyying
authored
merge

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix up code

Neehar Duvvuri
authored and
Neehar Duvvuri
committed
fix property
yingyingzhaozyying
committed
Jessli/convert1021 Fix and run black (#43571)

authored

Commits on Oct 22, 2025

fix merge conflict

Neehar Duvvuri
authored and
Neehar Duvvuri
committed