feat: Enable explanations even w/o explanation template #6190
Conversation
dosubot
bot
added
the
size:M
There was a problem hiding this comment.
I think we need to be a bit careful here, just adding text at the end forces us to be in charge of building the prompt - which might not be ideal in varying scenarios (different types of models, maybe the template already has system instructions to do this etc.).
| patched_explanation_template_text = ( | ||
| "*****\n\n" | ||
| "After following the previous instructions, add a paragraph that starts with " | ||
| "`EXPLANATION: ` and then provide a concise explanation of your reasoning." | ||
| ) |
There was a problem hiding this comment.
how do we know that the original template doesn't contain this instruction? I think there might be downsides to the fact that now the template is not declarative but constructed inside of the function itself?
The scenario I'm worried about is that Arize is now trying to build templates that are "evaluation" templates. Depending on how you pass it into this, the explanation postfix will now get added (possibly twice).
We should probably be careful here so I'm gonna block for now just to make sure we don't break any workflows.
There was a problem hiding this comment.
the postfix only gets added if the explanation template isn't provided, it's just fallback behavior
|
@mikeldking yeah we just don't know which is why we didn't implement something like this before. I wanted to just sketch this out in case we think on balance it's worth not simply failing out for custom templates passed to |
llm_classify,provide_explanationdoesn't do anything unless a user uses a customClassificationTemplateobject