Skip to content

feat(langfuse): tag traces with active profile - #26462

Closed
aqilaziz wants to merge 4 commits into
NousResearch:mainfrom
aqilaziz:feat-langfuse-profile-user-id
Closed

feat(langfuse): tag traces with active profile#26462
aqilaziz wants to merge 4 commits into
NousResearch:mainfrom
aqilaziz:feat-langfuse-profile-user-id

Conversation

@aqilaziz

Copy link
Copy Markdown
Contributor

Summary

  • thread the active Hermes profile into LLM and tool plugin hooks
  • set Langfuse trace user_id from the active profile and keep metadata.profile for redundancy
  • document per-profile Langfuse filtering in the bundled plugin README
  • add regression coverage for profile userId, run_agent hook payloads, and model_tools hook payloads
  • include CI stabilizers for Discord mock imports and Nous provider model defaults

Fixes #26455.

Tests

  • python -m pytest -o addopts= tests\plugins\test_langfuse_plugin.py::TestProfileUserId tests\run_agent\test_run_agent.py::TestRunConversation::test_request_scoped_api_hooks_fire_for_each_api_call tests\test_model_tools.py::TestHandleFunctionCall::test_tool_hooks_receive_session_and_tool_call_ids -q --tb=short
  • python -m pytest -o addopts= tests\plugins\test_langfuse_plugin.py -q --tb=short
  • python -m pytest -o addopts= tests\test_model_tools.py -q --tb=short
  • python -m pytest -o addopts= tests\run_agent\test_run_agent.py::TestRunConversation::test_request_scoped_api_hooks_fire_for_each_api_call -q --tb=short
  • python -m pytest -o addopts= tests\e2e\test_discord_adapter.py -q --tb=short
  • python -m pytest -o addopts= tests\run_agent\test_provider_parity.py::TestDeveloperRoleSwap::test_developer_role_via_nous_portal tests\run_agent\test_provider_parity.py::TestBuildApiKwargsNousPortal::test_includes_nous_product_tags tests\run_agent\test_provider_parity.py::TestBuildApiKwargsNousPortal::test_uses_chat_completions_format -q --tb=short
  • python -m ruff check plugins\observability\langfuse\__init__.py tests\plugins\test_langfuse_plugin.py run_agent.py model_tools.py hermes_cli\plugins.py tests\run_agent\test_run_agent.py tests\test_model_tools.py tests\e2e\conftest.py tests\run_agent\test_provider_parity.py
  • python -m py_compile plugins\observability\langfuse\__init__.py run_agent.py model_tools.py hermes_cli\plugins.py
  • git diff --check

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 15, 2026
@aqilaziz
aqilaziz force-pushed the feat-langfuse-profile-user-id branch from 38c81fc to e561a0c Compare May 15, 2026 22:42
@adoreparler

Copy link
Copy Markdown

Any luck with this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(langfuse): expose profile_name as trace userId for per-profile observability

3 participants