Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LangWatch Integration #2608

Merged
merged 15 commits into from
Jul 16, 2024
Merged

Add LangWatch Integration #2608

merged 15 commits into from
Jul 16, 2024

Conversation

rogeriochaves
Copy link
Contributor

@rogeriochaves rogeriochaves commented Jul 9, 2024

Hello there 👋

This PR adds LangWatch integration as a TracingService, I took the liberty of making some small improvements on the TracingService itself which helped getting a better logging, like passing the vertex as an argument so we can link hierarchically the components and masking the api_keys by default for security.

  • Add base LangWatch Integration
  • Convert inputs, outputs and error messages to have the best debugging experience on langwatch
  • Add parent-child hierarchy
  • Add docs
  • Tested with various workflows and fixed edge cases
  • Truncate input/output when it's too large for observability, like reading files for example

Some examples of it running:

Flow Trace
Two Outputs image image
Agent Tool Calling image image
Calling 3 LLMs to choose the best summary image image
Thread Visualization image image
Message list visualization image Table visualization Screenshot 2024-07-09 at 19 21 51

Then I'll leave somethings for next iterations which might take a bit longer:

  • Consider adding support for multiple parents on LangWatch, so we can represent it as a graph also on LangWatch side
  • Consider adding the langchain callback to track also the steps that happen inside the agents and other tools
  • Explore how we can capture explicitly the RAG contexts so RAG evaluations can be used on LangWatch side

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. documentation Improvements or additions to documentation enhancement New feature or request labels Jul 9, 2024
@rogeriochaves
Copy link
Contributor Author

rogeriochaves commented Jul 9, 2024

I'm having an issue with dependencies:

Because no versions of langwatch match >0.1.4,<0.2.0
 and langwatch (0.1.4) depends on pandas (>=2.2.2,<3.0.0), langwatch (>=0.1.4,<0.2.0) requires pandas (>=2.2.2,<3.0.0).
And because langflow-base (0.0.81) depends on pandas (2.2.0)
 and no versions of langflow-base match >0.0.81,<0.0.82, langwatch (>=0.1.4,<0.2.0) is incompatible with langflow-base (>=0.0.81,<0.0.82).
So, because langflow depends on both langflow-base (^0.0.81) and langwatch (^0.1.4), version solving failed.

however I did upgrade pandas to 2.2.2 already inside the backend/base/pyproject.toml, how can I proceed? Should I create another PR just bumping pandas version beforehand?

@ogabrielluiz ogabrielluiz changed the base branch from dev to main July 9, 2024 20:40
@ogabrielluiz
Copy link
Contributor

Hey @rogeriochaves

Thank you for this PR. This looks really awesome.

You had the wrong base branch. You should use the main branch for this. Could you merge your PR with the main branch?

@rogeriochaves
Copy link
Contributor Author

rogeriochaves commented Jul 10, 2024

@ogabrielluiz oh, sorry, I was told to use the dev branch, but now I see I opened a PR against main anyway, my bad then, will move over then!

@ogabrielluiz
Copy link
Contributor

@ogabrielluiz oh, sorry, I was told to use the dev branch, but now I see I opened a PR against main anyway, my bad then, will move over then!

I changed it for you. All you got to do is fix these conflicts and you should be good to go. Maybe rebase from main.

@rogeriochaves rogeriochaves force-pushed the dev branch 3 times, most recently from adb3112 to 4965c81 Compare July 11, 2024 15:53
@ogabrielluiz ogabrielluiz enabled auto-merge (squash) July 11, 2024 20:22
auto-merge was automatically disabled July 12, 2024 12:13

Head branch was pushed to by a user without write access

@rogeriochaves
Copy link
Contributor Author

@ogabrielluiz okay, pipeline is green now, ready to merge!

one note not related to the changes in this PR, but that I noticed after rebasing with main, now the tracing service only works when calling it through the API, no longer when running individual components or on the playground. Is that expected?

@ogabrielluiz
Copy link
Contributor

@ogabrielluiz okay, pipeline is green now, ready to merge!

one note not related to the changes in this PR, but that I noticed after rebasing with main, now the tracing service only works when calling it through the API, no longer when running individual components or on the playground. Is that expected?

No. It shouldn't be like that. I'll take a look

@rogeriochaves
Copy link
Contributor Author

@ogabrielluiz all good to merge again

…1, langwatch to version 0.1.3, and litellm to version 1.41.23
@ogabrielluiz ogabrielluiz enabled auto-merge (squash) July 16, 2024 17:28
@ogabrielluiz ogabrielluiz merged commit 89f2f62 into langflow-ai:main Jul 16, 2024
45 checks passed
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants