Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable test recordings for tests\inference, tests\telemetry and tests\connection #38475

Merged

Conversation

dargilco
Copy link
Member

Several changes as a result of getting all my tests pass from recordings on these three API surfaces.

  • Reduce the number of environment variables needed for connection tests, until I'm able to set up a more complex AI Studio for proper testing, where I will have multiple AOAI connections and AI Services connections.
  • Update dev_requirements.txt with additional dependencies needed to run inference tests
  • Remove sanitation of connection names, as those are actively used to verify tests pass/fail in connection tests. I don't think connection name by itself should be consider as a sensitive value that must be removed. But will need to talk to Ankit about it.
  • Add sanitizers for api-key and Application Insights connection strings in response from the service
  • Do not record chat completions call with AzureOpenAI, because that's a non-Azure SDK and does not support recording. Do the call only when you run live tests without recordings. Use recording base class utility function is_live_and_not_recording() to check recording status.
  • Remove tests that call project_client.telemetry.enable(destination="https://some.otlp.collector.endpoint") for now as I need to change them to run live against some local host service (3rd party tool that displays OTEL), and understand how that works.

Not in this PR: I updated the master copy of the .env file, and checked-in the recordings to the azure-sdk-for-python-pr repo.
See here: https://github.com/Azure/azure-sdk-for-python-pr/tree/feature/azure-ai-projects/sdk/ai/azure-ai-projects

@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-ai-projects

@dargilco dargilco merged commit 78d345d into feature/azure-ai-projects Nov 12, 2024
11 of 21 checks passed
@dargilco dargilco deleted the dargilco/azure-ai-projects-pr-11-11-0958 branch November 12, 2024 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants