Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support custom text formats and recursive #496

Merged
merged 36 commits into from
Nov 21, 2023
Merged

Support custom text formats and recursive #496

merged 36 commits into from
Nov 21, 2023

Conversation

thinkall
Copy link
Collaborator

@thinkall thinkall commented Oct 31, 2023

Why are these changes needed?

Address #408

Also, address #569

Moreover, made some other improvements during the implementation of this PR:

  • Fix missing search_string in update_context
  • Update docstring of docs_path
  • Update test for get_files_from_dir

Related issue number

Closes #408
Closes #569

Checks

@codecov-commenter
Copy link

codecov-commenter commented Oct 31, 2023

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (ef1c3d3) 29.75% compared to head (39e7718) 48.38%.

Files Patch % Lines
...gen/agentchat/contrib/retrieve_user_proxy_agent.py 55.55% 3 Missing and 1 partial ⚠️
autogen/retrieve_utils.py 50.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #496       +/-   ##
===========================================
+ Coverage   29.75%   48.38%   +18.63%     
===========================================
  Files          27       27               
  Lines        3455     3466       +11     
  Branches      782      826       +44     
===========================================
+ Hits         1028     1677      +649     
+ Misses       2353     1603      -750     
- Partials       74      186      +112     
Flag Coverage Δ
unittests 48.21% <58.82%> (+18.51%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@sonichi sonichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only minor suggestions. Again, please try to find a group of reviewers for RAG and frequently engage with them.

autogen/agentchat/contrib/retrieve_user_proxy_agent.py Outdated Show resolved Hide resolved
@sonichi sonichi added this pull request to the merge queue Nov 21, 2023
Merged via the queue into main with commit 07646d4 Nov 21, 2023
55 of 58 checks passed
@sonichi sonichi deleted the text_formats branch November 21, 2023 04:04
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* Add custom text types and recursive

* Add custom text types and recursive

* Fix format

* Update qdrant, Add pdf to unstructured

* Use unstructed as the default text extractor if installed

* Add tests for unstructured

* Update tests env for unstructured

* Fix error if last message is a function call, issue microsoft#569

* Remove csv, md and tsv from UNSTRUCTURED_FORMATS

* Update docstring of docs_path

* Update test for get_files_from_dir

* Update docstring of custom_text_types

* Fix missing search_string in update_context

* Add custom_text_types to notebook example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rag retrieve-augmented generative agents
Projects
None yet
6 participants