Skip to content

Issues: confident-ai/deepeval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

106 Open 187 Closed
106 Open 187 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Is there an initialisation bug in synthesizer/chunking/context_generator.py for class ContextGenerator method generate_contexts() ?
#1174 opened Nov 21, 2024 by CAW-nz
Issue regarding consistent loading of source_files data (from goldens original values)
#1173 opened Nov 21, 2024 by CAW-nz
Corrections for some of the DeepEval documentation/help pages
#1172 opened Nov 21, 2024 by CAW-nz
Bug: dataset.py - The two Open statements in EvaluationDataset.save_as() are lacking encoding="utf-8" argument
#1171 opened Nov 21, 2024 by CAW-nz
Use Poetry build instead of setup.py
#1169 opened Nov 19, 2024 by FrancoisMasson1990
How to integrate llama3 to deepeval?
#1163 opened Nov 15, 2024 by drbhushan
1
GEval not focusing on expected_output & Relying on OpenAI instead
#1149 opened Nov 11, 2024 by pavan-growexxer
2
Asynchronous test runs are sometimes not completed correctly
#1147 opened Nov 9, 2024 by jmaczan
12
Hallucination metric score is set to 0.0 in one run and 1.0 in another run, despite having the same input values
#1146 opened Nov 9, 2024 by jmaczan
1
Hallucination metric assigns only either 0.0 or 1.0 score
#1145 opened Nov 9, 2024 by jmaczan
1
Error while calculating Knowledge retention ; Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.
#1139 opened Nov 4, 2024 by jaysudhakaran
4
Improving Answer Relevancy Template
#1137 opened Nov 3, 2024 by dipanjanS
4
Get tokens for each request
#1114 opened Oct 22, 2024 by luqianjiang
1
TypeError while evaluating a test case for Role Adherence metric using CustomLlama3_8B.
#1105 opened Oct 21, 2024 by Qca-2023
HumanEval doesn't actually runs test
#1102 opened Oct 19, 2024 by cfytrok
The latest versions of deepeval and Instructor can't be installed together
#1100 opened Oct 18, 2024 by milest
5
APIConnectionError: Connection error.
#1094 opened Oct 17, 2024 by paulacanva
3
Accuracy Issue - accuracy is 0.00
#1090 opened Oct 17, 2024 by Pitambartech
Incompatibility with ragas v0.2.x causing TypeError in measure function
#1089 opened Oct 17, 2024 by lagom0327
Typo in scorer/scorer.py
#1087 opened Oct 16, 2024 by jtquach1
Missing API key and endpoint configuration...(etc) in OpenAI() and support for Azure OpenAI
#1086 opened Oct 16, 2024 by NaraSyuhei
1
Question About The Syntesizer
#1068 opened Oct 10, 2024 by Miriam2040
Results are not in the same order as input test cases
#1034 opened Sep 25, 2024 by joaopcm1996
1
Typo in docs/docs/benchmarks-introduction.mdx
#1032 opened Sep 24, 2024 by jtquach1
g_eval metric implicitly falls back to top score instead of sum of weighted top probabilities
#1029 opened Sep 23, 2024 by mattany
1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.