Lots of new features

Lots of new features this release:

1. JudgementalGPT now allows for different languages - useful for our APAC and European friends
2. RAGAS metrics now supports all OpenAI models - useful for those running into context length issues
3. LLMEvalMetric now returns a reasoning for its score
4. deepeval test run now has hooks that call on test run completion
5. evaluate now displays retrieval_context for RAG evaluation
6. RAGAS metric now displays metric breakdown for all its distinct metrics

Continuous Evaluation

Automatically integrated with Confident AI for continous evaluation throughout the lifetime of your LLM (app):

-log evaluation results and analyze metrics pass / fails
-compare and pick the optimal hyperparameters (eg. prompt templates, chunk size, models used, etc.) based on evaluation results
-debug evaluation results via LLM traces
-manage evaluation test cases / datasets in one place
-track events to identify live LLM responses in production
-add production events to existing evaluation datasets to strength evals over time

Continuous Evaluation

Automatically integrated with Confident AI for continous evaluation throughout the lifetime of your LLM (app):

-log evaluation results and analyze metrics pass / fails
-compare and pick the optimal hyperparameters (eg. prompt templates, chunk size, models used, etc.) based on evaluation results
-debug evaluation results via LLM traces
-manage evaluation test cases / datasets in one place
-track events to identify live LLM responses in production
-add production events to existing evaluation datasets to strength evals over time

Evaluate entire datasets

Mid-week bug fixes release with an extra feature:

• run_test now works
• new function evaluate, evaluates a list of test cases (dataset) on metrics you define, all without having to go through the CLI. More info here: https://docs.confident-ai.com/docs/evaluation-datasets#evaluate-your-dataset-without-pytest

Judgemental GPT

In this release, deepeval has added support for:

• JudgementalGPT, a dedicated LLM app developed by Confident AI to perform evaluations more robustly and accurately. JudgementalGPT provides a score and a reason for the score.
• Parallel testing: execute test cases in parallel and speed up evaluation up to 100x.

v0.20.17

new release

v0.20.16

new release

v0.20.15

new release

v0.20.14

prepare for release

v0.20.13

release v0.20.13