Add support for fine-grained tracing using W&B Weave

docs/source/guides/fine-grained-tracing-with-weave.md

@@ -0,0 +1,79 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Observing a Workflow with W&B Weave
+
+This guide provides a step-by-step process to enable observability in an AIQ Toolkit workflow using Weights and Biases (W&B) Weave for tracing using just a few lines of code in your workflow configuration file.
+
+![Weave Tracing Dashboard](../_static/weave_tracing.png)
+
+### Step 1: Install the Weave plugin
+
+To install the Weave plugin, run the following:
+
+```bash
+uv pip install -e '.[weave]'
+```
+
+### Step 2: Install the Workflow
+
+Pick an example from the list of available workflows. In this guide, we will be using the `simple_calculator` example.
+
+```bash
+uv pip install -e examples/simple_calculator
+```
+
+### Step 3: Modify Workflow Configuration
+
+Update your workflow configuration file to include the weave telemetry settings. For example, `examples/simple_calculator/configs/config-weave.yml` has the following weave settings:
+
+```bash
+general:
+  use_uvloop: true
+  telemetry:
+    tracing:
+      weave:
+        _type: weave
+        project: "aiqtoolkit-demo"
+```
+
+This setup enables logging trace data to W&B weave. The weave integration requires one parameter and one optional parameter:
+
+| Parameter | Description | Example |
+|-----------|-------------|---------|
+| `project` | The name of your W&B Weave project | `"aiqtoolkit-demo"` |
+| `entity` (optional) | Your W&B username or team name | `"your-wandb-username-or-teamname"` |
+
+### Step 4: Run Your Workflow
+From the root directory of the AIQ Toolkit library, execute your workflow as shown below:
+
+```bash
+aiq run --config_file examples/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?"
+```
+
+If it is your first time running the workflow, you will be prompted to login to W&B Weave.
+
+### Step 5: View Traces Data in Weave Dashboard
+
+As the workflow runs, you will find a Weave URL (starting with a 🍩 emoji). Click on the URL to access your logged trace timeline.
+
+Note how the integration captures not only the `aiq` intermediate steps but also the underlying framework. This is because [Weave has integrations](https://weave-docs.wandb.ai/guides/integrations/) with many of your favorite frameworks.
+
+## Resources
+
+- Learn more about tracing [here](https://weave-docs.wandb.ai/guides/tracking/tracing).
+- Learn more about how to navigate the logged traces [here](https://weave-docs.wandb.ai/guides/tracking/trace-tree).

docs/source/guides/index.md

@@ -25,6 +25,7 @@ Share Components <./sharing-workflows-and-tools.md>
 Evaluate <./evaluate.md>
 Add Custom Evaluators <./custom-evaluator.md>
 Evaluation Endpoints <./evaluate-api.md>
+./fine-grained-tracing-with-weave.md
 ./observe-workflow-with-phoenix.md
 Use User Interface and API Server <./using-aiqtoolkit-ui-and-server.md>
 MCP Server Front-End <./mcp-server.md>

examples/simple_calculator/src/aiq_simple_calculator/configs/config-weave.yml

@@ -0,0 +1,58 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+general:
+  use_uvloop: true
+  telemetry:
+    tracing:
+      weave:
+        _type: weave
+        project: "aiqtoolkit-demo"
+
+functions:
+  calculator_multiply:
+    _type: calculator_multiply
+  calculator_inequality:
+    _type: calculator_inequality
+  calculator_divide:
+    _type: aiq_simple_calculator/calculator_divide
+  current_datetime:
+    _type: current_datetime
+  calculator_subtract:
+    _type: calculator_subtract
+
+llms:
+  nim_llm:
+    _type: nim
+    model_name: meta/llama-3.1-70b-instruct
+    temperature: 0.0
+    max_tokens: 1024
+  openai_llm:
+    _type: openai
+    model_name: gpt-3.5-turbo
+    max_tokens: 2000
+
+workflow:
+  _type: react_agent
+  tool_names:
+    - calculator_multiply
+    - calculator_inequality
+    - current_datetime
+    - calculator_divide
+    - calculator_subtract
+  llm_name: nim_llm
+  verbose: true
+  retry_parsing_errors: true
+  max_retries: 3

packages/aiqtoolkit_weave/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools >= 64", "setuptools-scm>=8"]
+
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["aiq.*"]
+
+
+[tool.setuptools_scm]
+root = "../.."
+
+
+[project]
+name = "aiqtoolkit-weave"
+dynamic = ["version"]
+dependencies = [
+  # Keep package version constraints as open as possible to avoid conflicts with other packages. Always define a minimum
+  # version when adding a new package. If unsure, default to using `~=` instead of `==`. Does not apply to aiq packages.
+  # Keep sorted!!!
+  "aiqtoolkit",
+  "weave>=0.51.44"
+]
+requires-python = ">=3.12"
+description = "Subpackage for Weave integration in AIQToolkit"
+readme = "src/aiq/meta/pypi.md"
+keywords = ["ai", "observability", "wandb"]
+classifiers = ["Programming Language :: Python"]
+
+
+[tool.uv]
+config-settings = { editable_mode = "compat" }
+
+
+[tool.uv.sources]
+aiqtoolkit = { workspace = true }
+
+
+[project.entry-points.'aiq.components']
+aiqtoolkit_weave = "aiq.plugins.weave.register"

packages/aiqtoolkit_weave/src/aiq/meta/pypi.md

@@ -0,0 +1,23 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http:/www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+![NVIDIA Agent Intelligence Toolkit](https://media.githubusercontent.com/media/NVIDIA/AIQToolkit/refs/heads/main/docs/source/_static/aiqtoolkit_banner.png "AIQ Toolkit banner image"
+
+# NVIDIA Agent Intelligence Toolkit Subpackage
+This is a subpackage for Weights and Biases Weave integration for observability.
+
+For more information about AIQ Toolkit, please visit the [AIQ Toolkit package](https://pypi.org/project/aiqtoolkit/).

packages/aiqtoolkit_weave/src/aiq/plugins/weave/register.py

@@ -0,0 +1,20 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pylint: disable=unused-import
+# flake8: noqa
+# isort:skip_file
+
+from . import weave_sdk

packages/aiqtoolkit_weave/src/aiq/plugins/weave/weave_sdk.py

@@ -0,0 +1,49 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import Optional
+
+from pydantic import Field
+
+from aiq.builder.builder import Builder
+from aiq.cli.register_workflow import register_telemetry_exporter
+from aiq.data_models.telemetry_exporter import TelemetryExporterBaseConfig
+
+
+class WeaveTelemetryExporter(TelemetryExporterBaseConfig, name="weave"):
+    """A telemetry exporter to transmit traces to Weights & Biases Weave using OpenTelemetry."""
+    project: str = Field(description="The W&B project name.")
+    entity: Optional[str] = Field(default=None, description="The W&B username or team name.")
+
+
+@register_telemetry_exporter(config_type=WeaveTelemetryExporter)
+async def weave_telemetry_exporter(config: WeaveTelemetryExporter, builder: Builder):
+    import weave
+
+    if config.entity:
+        _ = weave.init(project_name=f"{config.entity}/{config.project}")
+    else:
+        _ = weave.init(project_name=config.project)
+
+    class NoOpSpanExporter:
+
+        def export(self, spans):
+            return None
+
+        def shutdown(self):
+            return None
+
+    # just yielding None errors with 'NoneType' object has no attribute 'export'
+    yield NoOpSpanExporter()

pyproject.toml

@@ -64,6 +64,7 @@ mem0ai = ["aiqtoolkit-mem0ai"]
 semantic-kernel = ["aiqtoolkit-semantic-kernel"]
 zep-cloud = ["aiqtoolkit-zep-cloud"]
 agno = ["aiqtoolkit-agno"]
+weave = ["aiqtoolkit-weave"]
 
 examples = [
   "aiq_email_phishing_analyzer",
@@ -98,6 +99,7 @@ aiqtoolkit-semantic-kernel = { workspace = true }
 aiqtoolkit-test = { workspace = true }
 aiqtoolkit-zep-cloud = { workspace = true }
 aiqtoolkit-agno = { workspace = true }
+aiqtoolkit-weave = { workspace = true }
 
 # All examples here
 aiq_email_phishing_analyzer = { path = "examples/email_phishing_analyzer", editable = true }