Hello，calculate faithfulness Error: No statements were generated from the answer #1048
Comments
|
Hello, @13416157913! I'm here to help you with any bugs, questions, or contributions. To resolve the "No statements were generated from the answer" error and the OpenAI API connection error when calculating the faithfulness metric using the Ragas library with Chinese text, you need to adapt the prompts to the Chinese language. The Here is how you can adapt the # Assuming `faithfulness` is an instance of the Faithfulness class
faithfulness.adapt(language="chinese")This will adapt the For the import os
import time
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
from openai.error import RateLimitError, APIError
os.environ["OPENAI_API_KEY"] = "your-openai-key"
data_samples = {
'question': [''],
'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
'contexts': [['']],
'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}
dataset = Dataset.from_dict(data_samples)
def evaluate_with_retries(dataset, metrics, retries=3, delay=5):
for attempt in range(retries):
try:
score = evaluate(dataset, metrics=metrics)
return score.to_pandas()
except (RateLimitError, APIError) as e:
if attempt < retries - 1:
time.sleep(delay)
delay *= 2 # Exponential backoff
else:
raise e
score_df = evaluate_with_retries(dataset, metrics=[answer_similarity, faithfulness, answer_correctness])
print(score_df)This code will retry the evaluation up to three times with an exponential backoff delay if a |
hello,when I set faithfulness.adapt(language="chinese") in my code, I meet another error, |
|
To resolve the
Here is an example: from ragas.llms import LangchainLLMWrapper
from langchain_openai.chat_models import ChatOpenAI
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
from datasets import Dataset
import os
os.environ["OPENAI_API_KEY"] = "your-openai-key"
# Initialize your LLM
llm_instance = LangchainLLMWrapper(ChatOpenAI())
# Set the LLM for the faithfulness metric
faithfulness.llm = llm_instance
# Adapt the faithfulness metric to Chinese
faithfulness.adapt(language="chinese")
data_samples = {
'question': [''],
'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
'contexts': [['']],
'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}
dataset = Dataset.from_dict(data_samples)
# Evaluate the dataset
score = evaluate(dataset, metrics=[answer_similarity, faithfulness, answer_correctness])
print(score.to_pandas())This ensures that the |
Hello, I use your solution, add code like this: os.environ["OPENAI_API_KEY"] = "xxxxxx" Initialize your LLMllm_instance = LangchainLLMWrapper(ChatOpenAI()) Set the LLM for the faithfulness metricfaithfulness.llm = llm_instance Adapt the faithfulness metric to Chinesefaithfulness.adapt(language="chinese") data_samples = { dataset = Dataset.from_dict(data_samples) score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness]) also meet the error: No statements were generated from the answer. Evaluating: 33%|███▎ | 1/3 [00:01<00:03, 1.81s/it]No statements were generated from the answer. [1 rows x 7 columns] Process finished with exit code 0 |
|
To resolve the issue of "No statements were generated from the answer" when calculating the faithfulness metric using the Ragas library with Chinese text, you need to ensure that the Here is the relevant part of the code where the def __post_init__(self):
if self.sentence_segmenter is None:
language = self.nli_statements_message.language
self.sentence_segmenter = get_segmenter(language=language, clean=False)To handle Chinese text, you need to make sure that the Here is an example of how you might modify the def __post_init__(self):
if self.sentence_segmenter is None:
language = self.nli_statements_message.language
if language == "chinese":
self.sentence_segmenter = ChineseSegmenter() # Replace with actual Chinese segmenter
else:
self.sentence_segmenter = get_segmenter(language=language, clean=False)Make sure to replace |
[ ]Hello I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
Hello, when I run a sample like this with chinese:
from datasets import Dataset
import os
from ragas import evaluate
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
os.environ["OPENAI_API_KEY"] ="xxxxxx"
data_samples = {
'question': [''],
'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. 安全措施：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. 检查LED日光灯：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. 检查电压：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. 检查供电箱和控制空开：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. 检查卡位和安装：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. 检查接线：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. 重新接通电源并测试：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
'contexts' : [['']],
'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}
dataset = Dataset.from_dict(data_samples)
score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness])
print (score["answer_correctness"])
print(score.to_pandas())
Ragas version: 0.1.9
Python version: 3.10
Code to Reproduce
Share code to reproduce the issue
Error trace
Evaluating: 0%| | 0/3 [00:00<?, ?it/s]No statements were generated from the answer.
Evaluating: 67%|██████▋ | 2/3 [01:15<00:37, 37.71s/it]
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 371, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 268, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 251, in handle_async_request
response = await connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 133, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 111, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 176, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 226, in _receive_event
raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1522, in _request
response = await self._client.send(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1646, in send
response = await self._send_handling_auth(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1674, in _send_handling_auth
response = await self._send_handling_redirects(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1711, in _send_handling_redirects
response = await self._send_single_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1748, in _send_single_request
response = await transport.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 370, in handle_async_request
with map_httpcore_exceptions():
File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 371, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 268, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 251, in handle_async_request
response = await connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 133, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 111, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 176, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 226, in _receive_event
raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1522, in _request
response = await self._client.send(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1646, in send
response = await self._send_handling_auth(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1674, in _send_handling_auth
response = await self._send_handling_redirects(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1711, in _send_handling_redirects
response = await self._send_single_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1748, in _send_single_request
response = await transport.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 370, in handle_async_request
with map_httpcore_exceptions():
File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 371, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 268, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection_pool.py", line 251, in handle_async_request
response = await connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 133, in handle_async_request
raise exc
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 111, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 176, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore_async\http11.py", line 226, in _receive_event
raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1522, in _request
response = await self._client.send(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1646, in send
response = await self._send_handling_auth(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1674, in _send_handling_auth
response = await self._send_handling_redirects(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1711, in _send_handling_redirects
response = await self._send_single_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1748, in _send_single_request
response = await transport.handle_async_request(request)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 370, in handle_async_request
with map_httpcore_exceptions():
File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\ProgramData\anaconda3\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 95, in run
results = self.loop.run_until_complete(self._aresults())
File "D:\ProgramData\anaconda3\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 83, in _aresults
raise e
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 78, in aresults
r = await future
File "D:\ProgramData\anaconda3\lib\asyncio\tasks.py", line 571, in wait_for_one
return f.result() # May raise f.exception().
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 37, in sema_coro
return await coro
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 111, in wrapped_callable_async
return counter, await callable(*args, **kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\base.py", line 125, in ascore
raise e
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\base.py", line 121, in ascore
score = await self.ascore(row=row, callbacks=group_cm, is_async=is_async)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics_answer_correctness.py", line 253, in ascore
is_statement_present = await self.llm.generate(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\llms\base.py", line 93, in generate
return await agenerate_text_with_retry(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio_init.py", line 189, in async_wrapped
return await copy(fn, *args, **kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio_init.py", line 111, in call
do = await self.iter(retry_state=retry_state)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio_init.py", line 153, in iter
result = await action(retry_state)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity_utils.py", line 99, in inner
return call(*args, **kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity_init.py", line 398, in
self._add_action_func(lambda rs: rs.outcome.result())
File "D:\ProgramData\anaconda3\lib\concurrent\futures_base.py", line 451, in result
return self.__get_result()
File "D:\ProgramData\anaconda3\lib\concurrent\futures_base.py", line 403, in __get_result
raise self.exception
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio_init.py", line 114, in call
result = await fn(*args, **kwargs)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\llms\base.py", line 170, in agenerate_text
return await self.langchain_llm.agenerate_prompt(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 609, in agenerate_prompt
return await self.agenerate(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 569, in agenerate
raise exceptions[0]
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 754, in _agenerate_with_cache
result = await self._agenerate(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_openai\chat_models\base.py", line 666, in _agenerate
response = await self.async_client.create(messages=message_dicts, **params)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\resources\chat\completions.py", line 1214, in create
return await self._post(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1790, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1493, in request
return await self._request(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1546, in _request
return await self._retry_request(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1615, in _retry_request
return await self._request(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1546, in _request
return await self._retry_request(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1615, in _retry_request
return await self._request(
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai_base_client.py", line 1556, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
Traceback (most recent call last):
File "E:\PCI-Agent\ragas_examples.py", line 25, in
score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness])
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\evaluation.py", line 250, in evaluate
raise e
File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\evaluation.py", line 232, in evaluate
raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass
raise_exceptions=Falseincase you want to show only a warning message instead.Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: