chore: extend accuracy tests to support custom user and cluster config #655
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This PR extends the accuracy testing framework to support custom user configurations and cluster configurations, enabling more flexible test scenarios including vector search functionality.
Key changes:
- Modified
describeAccuracyTeststo accept optionaluserConfigandclusterConfigparameters - Updated the accuracy testing client to use a generic user config object instead of individual API parameters
- Enhanced MongoDB cluster process to include
directConnection=truein connection strings
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/accuracy/sdk/describeAccuracyTests.ts | Extended function signature to accept user and cluster configurations, refactored to use generic config handling |
| tests/accuracy/sdk/accuracyTestingClient.ts | Simplified initialization to accept a generic user config object instead of individual parameters |
| tests/accuracy/createIndex.test.ts | Updated to use new API with explicit user config and cluster config for vector search testing |
| tests/integration/tools/mongodb/mongodbClusterProcess.ts | Added directConnection=true to MongoDB connection string for better cluster compatibility |
himanshusinghs
added
the
no-title-validation
himanshusinghs
changed the title
This comment has been minimized.
This comment has been minimized.
Extends describeAccuracyTests interface to accept user config and cluster config to allow for running accuracy tests against different cluster configuration and user configuration.
himanshusinghs
force-pushed
the
chore/extend-accuracy-tests
branch
from
4efecc2 to
e83d9a2
Compare
📊 Accuracy Test Results📈 Summary
📊 Baseline Comparison
📎 Download Full HTML Report - Look for the Report generated on: 10/15/2025, 5:54:12 PM |
nirinchev
approved these changes
Oct 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
This PR extends describeAccuracyTests interface to accept user config and cluster config to allow for running accuracy tests against different cluster configuration and user configuration.
This will come in handy to write accuracy tests against feature flags and different cluster configurations.
Checklist