Skip to content

Refactor code and documentations - #5

Merged
nhungbi merged 1 commit into
mainfrom
squashed_refactoring
Aug 19, 2025
Merged

Refactor code and documentations#5
nhungbi merged 1 commit into
mainfrom
squashed_refactoring

Conversation

@nhungbi

@nhungbi nhungbi commented Aug 18, 2025

Copy link
Copy Markdown
Contributor

Issue #, if available:

Description of changes:

Documentations:

  • Updated README with new features/import shortcuts.
  • Updated pyproject to use strands-agents-evals instead of strands-evaluation
  • Finalized examples for basic use cases in /examples (reorder steps so that task definition is first, updated imports, and got rid of redundant examples), also added examples for generating datasets

Structure:

  • Moved files in utils to new folders and get rid of /utils. Helper functions to extract information in tasks are now in /extractors
  • Changed the name of the main package from strands_evaluation to strands_evals
  • Filled out init.py for nicer import shortcuts
  • Got rid of mutable types as defaults
  • Got rid of name == "main"
  • Restructured /tests to have the same directory structure as /src and rename the tests to have the same format as strands

Code Logic:

  • Added exception in _run_task in Dataset for not supporting async task function in synchronous _run_task
  • Allowed user to return "input" in the task function to update the "input" for evaluation (will leave the original test case alone) for multi-shots or similar scenarios evaluation
  • Got rid of @DataClass for Evaluator and BaseModel in Dataset
  • In InteractionEvaluator, the prompt will now only include the input for the first interaction evaluation or iteratively select the correct input if the input is a list that matches the number of interactions
  • Added TaskOutput typedDict
  • Got rid of list[str] and use generic list in Interaction class to allow for more flexible definition of Interaction

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@nhungbi
nhungbi requested a review from poshinchen August 18, 2025 15:37
@nhungbi
nhungbi merged commit 8812c38 into main Aug 19, 2025
@nhungbi
nhungbi deleted the squashed_refactoring branch August 19, 2025 13:36
kylehounslow added a commit to kylehounslow/strands-agents-evals that referenced this pull request Apr 9, 2026
…tests

- Add constructor docstring with Args and usage examples (strands-agents#7)
- Clean up auth type hint to just Any (strands-agents#6)
- Re-raise ProviderError before broad except catch (strands-agents#3)
- Add _parse_time edge case tests: empty, None, invalid, Z suffix, offset (strands-agents#4)
- Add multi-trace provider test verifying flatten/regroup (strands-agents#5)
- 24 tests total
kylehounslow added a commit to kylehounslow/strands-agents-evals that referenced this pull request Apr 10, 2026
…tests

- Add constructor docstring with Args and usage examples (strands-agents#7)
- Clean up auth type hint to just Any (strands-agents#6)
- Re-raise ProviderError before broad except catch (strands-agents#3)
- Add _parse_time edge case tests: empty, None, invalid, Z suffix, offset (strands-agents#4)
- Add multi-trace provider test verifying flatten/regroup (strands-agents#5)
- 24 tests total
@strandly-the-agent strandly-the-agent mentioned this pull request Jul 25, 2026
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants