Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 0.0.2 of Autogenbench #1548

Merged
merged 20 commits into from
Feb 24, 2024
Merged

Version 0.0.2 of Autogenbench #1548

merged 20 commits into from
Feb 24, 2024

Conversation

afourney
Copy link
Member

@afourney afourney commented Feb 5, 2024

Why are these changes needed?

Quality of life improvements for AutoGenBench. Fixes (so far) include:

  1. Printing and logging of the AutoGenBench version (not just pyautogen's version)
  2. Automatic discovery of Templates when running init_tasks. (Thanks @gagb)
  3. Detection and use of @cheng-tan 's logging library, when available (Logging #1146 )
  4. (Experimental) Ability for tasks to install pyautogen from a local folder (mounted as a volume in Docker)
  5. (Experimental) Automatic discovery of the above-mentioned local folder, when running AutoGenBench within a subfolder of the repo
  6. (Experimental) Use of venvs when running tasks natively (brings the experience closer to running with Docker. Provides more isolation)
  7. Ability to set an environment variable to bypass the need for manual acknowledgment of the warning when running natively (Use with caution!)

Checks

@afourney afourney added the proj-autogenbench Issues related to AutoGenBench. label Feb 5, 2024
@afourney afourney self-assigned this Feb 5, 2024
@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7ceee5d) 39.62% compared to head (09237cc) 39.62%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1548   +/-   ##
=======================================
  Coverage   39.62%   39.62%           
=======================================
  Files          57       57           
  Lines        6006     6006           
  Branches     1338     1338           
=======================================
  Hits         2380     2380           
  Misses       3433     3433           
  Partials      193      193           
Flag Coverage Δ
unittests 39.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@afourney afourney marked this pull request as ready for review February 16, 2024 06:38
@sonichi sonichi added this pull request to the merge queue Feb 24, 2024
Merged via the queue into main with commit 085bf6c Feb 24, 2024
22 checks passed
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* Prints the version of AutoGenBench from the command line, closing i1458

* Added autogenbench version to timestamp.txt

* Attempting to fix formatting.

* Add a gitignore for autogenbench

* Generalize to read all template dirs from Templates

* AutoGenBench logs telemetry when available.

* Remove spaces if present from template names.

* Bump version.

* Fixed formatting.

* Allow native warning to be skipped. Mount autogen repo in Docker if it can be found (experimental).

* Native execution now occurs in a venv.

* Bump version.

* Fixed a prompt escaping bug evident in GAIA task '6f37996b-2ac7-44b0-8e68-6d28256631b4'

* Updated all scenarios to use template discovery.

* Update with main version of runtime_logging.

---------

Co-authored-by: gagb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proj-autogenbench Issues related to AutoGenBench.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants