[integration]: Deepeval logger - #10382
spike-spiegel-21 wants to merge 11 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
|
|
Hi @spike-spiegel-21, please do not delete our contribution guidelines. Pre-Submission checklistPlease complete all items before asking a LiteLLM maintainer to review your PR
|
|
Hey @krrishdholakia, Apologies. We have accommodated all the contribution guidelines. |
| # Class variables or attributes | ||
| def __init__(self) -> None: | ||
| try: | ||
| from deepeval.tracing.tracing import ToolAttributes |
There was a problem hiding this comment.
Hey @spike-spiegel-21 can we please use an httpx implementation here instead.
It will be difficult for proxy users to use deepeval, if it requires additional dependencies.
There was a problem hiding this comment.
if deepeval is otel compatible, you can implement it similar to arize -
There was a problem hiding this comment.
hey @krrishdholakia, nice observation. currently we do not have a dedicated httpx implementation. However, we have mentioned the dependency requirement in the proxy docs.
No, we are not OTEL compatible at the moment.
|
closing this as planned in this PR: #10649 |
This branch was successfully deployed
2 participants
Deepeval (Confidential AI) integration with LiteLLM's trace callback
DeepEval is an open-source evaluation framework for LLMs (Github).
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]Type
🆕 New Feature
Changes
Screenshot for new test passing locally:
