Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Context Precision Example #1036

Open
MatthewSH opened this issue Jun 20, 2024 · 1 comment
Open

Context Precision Example #1036

MatthewSH opened this issue Jun 20, 2024 · 1 comment
Labels
question Further information is requested

Comments

@MatthewSH
Copy link

MatthewSH commented Jun 20, 2024

[x] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question
I was reviewing the context precision example on the website and noticed that answer is being provided as an input even though it's not being used actively as far as we can tell in the context precision metric. We even see a couple spots where the answer key is being overwritten with the ground_truth key.

My question is, for the examples are they supposed to be minimal and required arguments or just a data dump of everything that can be provided?

I know that using answer is going to be deprecated soon and according to https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_context_precision.py#L117 it will fall back, I was just curious if it having both on the example is intended or not.

EDIT: Also if the answer version is being deprecated soon, is the plan to switch all references from answer to ground_truth?

@MatthewSH MatthewSH added the question Further information is requested label Jun 20, 2024
Copy link

dosubot bot commented Jun 20, 2024

Hello @MatthewSH! I'm a bot here to help you with any issues or questions you have. I can assist with bug fixes, answer inquiries, and guide you on becoming a contributor. How can I assist you today?

The examples for context precision are supposed to be minimal and required arguments, not a data dump of everything that can be provided. The provided examples focus on the necessary inputs: question, context, and answer, and demonstrate how the context was useful in arriving at the given answer [1][2].

To continue talking to Dosu, mention @dosu.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant