Skip to content

Conversation

@joseharriaga
Copy link
Collaborator

From @trrwilson:

  • An informal "smoke test" category is added and applied to tests that are extremely fast and require no auth/credentials -- perfect for ensuring client creation, serialization, etc. work
  • A couple of new mock chat tests -- including one that would have caught last weekend's regression -- are included in this category
  • The "smoke" category is added to the CI workflow; this only adds 2-3s to the test task
  • NUnit Parallelization is enabled (degree: 8) to speed up live test runs; this currently reduces a ~10m run to ~1.5m
    • As written, we're bottlenecked by Assistants and VectorStore tests, as they exercise several stateful assumptions and have to be kept serial with respect to themselves; this is likely unnecessarily fragile to begin with and can be fixed with rewrites later
    • I didn't encounter any rate limit issues, but it's easy to adjust the degree of parallelism and/or per-fixture parallelization as needed

@joseharriaga joseharriaga merged commit 3e3bbfc into main Jul 11, 2024
@joseharriaga joseharriaga deleted the joseharriaga/Add-SmokeCategory branch July 11, 2024 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants