Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Corrections for some of the DeepEval documentation/help pages #1172

Open
CAW-nz opened this issue Nov 21, 2024 · 0 comments
Open

Corrections for some of the DeepEval documentation/help pages #1172

CAW-nz opened this issue Nov 21, 2024 · 0 comments

Comments

@CAW-nz
Copy link

CAW-nz commented Nov 21, 2024

@penguine-ip

There are two separate pages that need to be corrected + one possible addition to cover off an undocumented parameter:

The example under https://docs.confident-ai.com/docs/synthesizer-introduction#filteration-quality
is wrongly referencing Synthesizer(filteration_config=...) when it should be filtration_config. That is removal of "e" in the spelling.

And the object instance probably should consistently be renamed accordingly (to avoid naming confusion).
The full example would become:

from deepeval.synthesizer.config import FiltrationConfig

filtration_config = FiltrationConfig(
  critic_model="gpt-4o",
  synthetic_input_quality_threshold=0.5
)

synthesizer = Synthesizer(filtration_config=filtration_config)

Finally the Title of this section and URL reference to jump to this section should be updated accordingly from:
https://docs.confident-ai.com/docs/synthesizer-introduction#filteration-quality
to
https://docs.confident-ai.com/docs/synthesizer-introduction#filtration-quality
(or similar)


The section under https://docs.confident-ai.com/docs/synthesizer-generate-from-docs#customize-context-construction
wrongly references ContextExtractionConfig whereas the actual class in synthesizer/config.py has defined the
class ContextConstructionConfig.

The example and the two related text references in this section need to be updated (eg see the first sentence of this section).
NB: Minor but there is also a typo in the example saying "from deepeval.synthesizer.confg ..." instead of "config".

Summary of updates needed:
The two class name references in this section should be updated, and the full example should be updated from:

from deepeval.synthesizer.confg import ContextExtractionConfig

...
synthesizer.generate_goldens_from_docs(
  document_paths=['example.txt', 'example.docx', 'example.pdf'],
  context_extraction_config=ContextExtractionConfig()
)

To:

from deepeval.synthesizer.config import ContextConstructionConfig

...
synthesizer.generate_goldens_from_docs(
  document_paths=['example.txt', 'example.docx', 'example.pdf'],
  context_construction_config=ContextConstructionConfig()
)

There is an undocumented optional parameter for generate_goldens_from_contexts method being source_files that you might like to now document under the relevant section.
URL = https://docs.confident-ai.com/docs/synthesizer-generate-from-contexts#generate-your-goldens

Could add new bullet point:

  • [Optional] source_files: a list of strings being the source of the contexts. Length must be aligned with the length of contexts.

(or similar wording)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant