chore(test): fix flaky telemetry tests

[Elbehery]

What does this PR do?

This PR fixes flaky telemetry tests

See #2814

Test Plan

[Elbehery]

cc @cdoern

[cdoern]

thanks! Is there explanation as to why this is the proper fix?

[Elbehery]

thanks! Is there explanation as to why this is the proper fix?

so the CI failed on with this error

pydantic_core._pydantic_core.ValidationError: 1 validation error for Trace
root_span_id
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]

the failure is due to database can contain traces where root_span_id is NULL (when no span was marked with the LOCAL_ROOT_SPAN_MARKER), but the Trace Pydantic model expects root_span_id to be a required string field.

[Elbehery]

ci is green 🙏🏽

[ehhuang]

don't think this is needed?

[ehhuang]

Just the filter below should be enough?

[Elbehery]

done 👍🏽

[leseb]

was this actually addressed?

[Elbehery]

i have explained below in comments

• chore(test): fix flaky telemetry tests #2815 (comment)
• chore(test): fix flaky telemetry tests #2815 (comment)

[Elbehery]

@ehhuang the failure did not happen last run

lets try keeping COALESCE

[ehhuang]

@ehhuang the failure did not happen last run

lets try keeping COALESCE

I'd prefer that we don't introduce unnecessary changes unless there's a reason to.

[Elbehery]

@ehhuang the failure did not happen last run
lets try keeping COALESCE

I'd prefer that we don't introduce unnecessary changes unless there's a reason to.

the CI failed after removing COALESCE https://github.com/meta-llama/llama-stack/actions/runs/16382156494/job/46295644087?pr=2815

[Elbehery]

@ehhuang the failure did not happen last run
lets try keeping COALESCE

I'd prefer that we don't introduce unnecessary changes unless there's a reason to.

the CI failed after removing COALESCE https://github.com/meta-llama/llama-stack/actions/runs/16382156494/job/46295644087?pr=2815

no failure with COALESCE :D

[Elbehery]

Another question, this repo always request to update branch, even if the files changed in PR are not changed !!

It is big effort to keep rebasing after each PR merge :D

[Elbehery]

I think requiring

Require linear history

it too strict, as long as no conflicts in PRs

cc @leseb

[Elbehery]

the failures are from #2575

This PR used @pytest.mark.asyncio

cc @leseb

[ashwinb]

Hey @Elbehery @ehhuang -- can we please DISABLE telemetry tests ASAP first and then land this PR? I realize this PR makes some changes which need careful consideration so while we iterate we need CI to be green.

[ehhuang]

#2814 @ashwinb

[Elbehery]

@ehhuang the ci failed again, it passes only with COALESCE

[ashwinb]

#2814 @ashwinb

thank you, merged that one.

[Elbehery]

@ehhuang the ci failed again, it passes only with COALESCE

@ehhuang shall we keep COALESCE ?

[ehhuang]

@ehhuang the failure did not happen last run
lets try keeping COALESCE

I'd prefer that we don't introduce unnecessary changes unless there's a reason to.

the CI failed after removing COALESCE https://github.com/meta-llama/llama-stack/actions/runs/16382156494/job/46295644087?pr=2815

looks like the revert wasn't clean (root_span_id was removed)

I think you removed t.root_span_id and that's why the tests fail without COALESCE

[Elbehery]

@ehhuang any other changes required ?

[ehhuang]

Thank you for working on this!

Can you remove the @pytest.mark.skip(reason="Skipping telemetry tests for now") too?

[Elbehery]

Thank you for working on this!

Can you remove the @pytest.mark.skip(reason="Skipping telemetry tests for now") too?

i was thinking to add them already, but thought maybe I should wait till you request :D