Forward-merge release/1.3 into develop (#880)

Forward-merge triggered by push to release/1.3 that creates a PR to keep
develop up-to-date. If this PR is unable to be immediately merged due to
conflicts, it will remain open for the team to manually merge. See
[forward-merger
docs](https://docs.rapids.ai/maintainers/forward-merger/) for more info.

Author: dagardner-nv

.coderabbit.yaml

@@ -28,6 +28,7 @@ reviews:
     issue_assessment:
       mode: warning
   auto_review:
+    base_branches: ["develop", "release/.*"]
     enabled: true
     drafts: false
     auto_incremental_review: true

README.md

@@ -81,13 +81,13 @@ pip install nvidia-nat
 NeMo Agent Toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain/LangGraph plugin, run the following:
 
 ```bash
-pip install nvidia-nat[langchain]
+pip install "nvidia-nat[langchain]"
 ```
 
 Or for all optional dependencies:
 
 ```bash
-pip install nvidia-nat[all]
+pip install "nvidia-nat[all]"
 ```
 
 The full list of optional dependencies can be found [here](./docs/source/quick-start/installing.md#framework-integrations).

ci/scripts/gitlab/report_test_results.py

@@ -99,6 +99,7 @@ def add_text(text: str, blocks: list[dict], plain_text: list[str]) -> None:
 
 
 def build_messages(junit_data: dict[str, typing.Any], coverage_data: str) -> ReportMessages:
+    branch_name = os.environ.get("CI_COMMIT_BRANCH", "unknown")
     num_errors = junit_data['num_errors']
     num_failures = junit_data['num_failures']
 
@@ -107,7 +108,7 @@ def build_messages(junit_data: dict[str, typing.Any], coverage_data: str) -> Rep
     plain_text = []
     blocks = []
 
-    summary_line = f"Nightly CI/CD Test Results for {date.today()}"
+    summary_line = f"Nightly CI/CD Test Results for `{branch_name}` - {date.today()}"
     plain_text.append(summary_line + "\n")
 
     num_errors_and_failures = num_errors + num_failures

ci/vale/styles/config/vocabularies/nat/accept.txt

@@ -102,6 +102,7 @@ onboarding
 [Oo]verfitting
 pandas
 [Pp]arallelization
+Pareto
 [Pp]arsable
 Patronus
 PCIe

docs/source/extend/telemetry-exporters.md

@@ -266,14 +266,14 @@ Before creating a custom exporter, check if your observability service is alread
 | Service | Type | Installation | Configuration |
 |---------|------|-------------|---------------|
 | **File** | `file` | `pip install nvidia-nat` | local file or directory |
-| **Langfuse** | `langfuse` | `pip install nvidia-nat[opentelemetry]` | endpoint + API keys |
-| **LangSmith** | `langsmith` | `pip install nvidia-nat[opentelemetry]` | endpoint + API key |
-| **OpenTelemetry Collector** | `otelcollector` | `pip install nvidia-nat[opentelemetry]` | endpoint + headers |
-| **Patronus** | `patronus` | `pip install nvidia-nat[opentelemetry]` | endpoint + API key |
-| **Galileo** | `galileo` | `pip install nvidia-nat[opentelemetry]` | endpoint + API key |
-| **Phoenix** | `phoenix` | `pip install nvidia-nat[phoenix]` | endpoint |
-| **RagaAI/Catalyst** | `catalyst` | `pip install nvidia-nat[ragaai]` | API key + project |
-| **Weave** | `weave` | `pip install nvidia-nat[weave]` | project name |
+| **Langfuse** | `langfuse` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API keys |
+| **LangSmith** | `langsmith` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API key |
+| **OpenTelemetry Collector** | `otelcollector` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + headers |
+| **Patronus** | `patronus` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API key |
+| **Galileo** | `galileo` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API key |
+| **Phoenix** | `phoenix` | `pip install "nvidia-nat[phoenix]"` | endpoint |
+| **RagaAI/Catalyst** | `catalyst` | `pip install "nvidia-nat[ragaai]"` | API key + project |
+| **Weave** | `weave` | `pip install "nvidia-nat[weave]"` | project name |
 
 ### Simple Configuration Example
 
@@ -412,7 +412,7 @@ class CustomSpanExporter(SpanExporter[Span, dict]):
 > **Note**: OpenTelemetry exporters require the `nvidia-nat-opentelemetry` subpackage. Install it with:
 
 > ```bash
-> pip install nvidia-nat[opentelemetry]
+> pip install "nvidia-nat[opentelemetry]"
 > ```
 
 For most OTLP-compatible services, use the pre-built `OTLPSpanAdapterExporter`:

docs/source/quick-start/installing.md

@@ -92,13 +92,13 @@ pip install nvidia-nat
 NeMo Agent toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain/LangGraph plugin, run the following:
 
 ```bash
-pip install nvidia-nat[langchain]
+pip install "nvidia-nat[langchain]"
 ```
 
 Or for all optional dependencies:
 
 ```bash
-pip install nvidia-nat[all]
+pip install "nvidia-nat[all]"
 ```
 
 The full list of optional dependencies can be found [here](../quick-start/installing.md#framework-integrations).

docs/source/reference/api-server-endpoints.md

@@ -61,7 +61,7 @@ result back to the client. The transaction schema is defined by the workflow.
 ## Asynchronous Generate
 The asynchronous generate endpoint allows clients to submit a workflow to run in the background and return a response immediately with a unique identifier for the workflow. This can be used to query the status and results of the workflow at a later time. This is useful for long-running workflows, which would otherwise cause the client to time out.
 
-This endpoint is only available when the `async_endpoints` optional dependency extra is installed. For users installing from source, this can be done by running `uv pip install -e '.[async_endpoints]'` from the root directory of the NeMo Agent toolkit library. Similarly, for users installing from PyPI, this can be done by running `pip install 'nvidia-nat[async_endpoints]'`.
+This endpoint is only available when the `async_endpoints` optional dependency extra is installed. For users installing from source, this can be done by running `uv pip install -e '.[async_endpoints]'` from the root directory of the NeMo Agent toolkit library. Similarly, for users installing from PyPI, this can be done by running `pip install "nvidia-nat[async_endpoints]"`.
 
 Asynchronous jobs are managed using [Dask](https://docs.dask.org/en/stable/). By default, a local Dask cluster is created at start time, however you can also configure the server to connect to an existing Dask scheduler by setting the `scheduler_address` configuration parameter. The Dask scheduler is used to manage the execution of asynchronous jobs, and can be configured to run on a single machine or across a cluster of machines. Job history and metadata is stored in a SQL database using [SQLAlchemy](https://www.sqlalchemy.org/). By default, a temporary SQLite database is created at start time, however you can also configure the server to use a persistent database by setting the `db_url` configuration parameter. Refer to the [SQLAlchemy documentation](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls) for the format of the `db_url` parameter. Any database supported by [SQLAlchemy's Asynchronous I/O extension](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html) can be used. Refer to [SQLAlchemy's Dialects](https://docs.sqlalchemy.org/en/20/dialects/index.html) for a complete list (many but not all of these support Asynchronous I/O).
 

docs/source/reference/evaluate-api.md

@@ -20,7 +20,7 @@ limitations under the License.
 It is recommended that the [Evaluating NeMo Agent toolkit Workflows](./evaluate.md) guide be read before proceeding with this detailed documentation.
 :::
 
-The evaluation endpoint can be used to start evaluation jobs on a remote NeMo Agent toolkit server. This endpoint is only available when the `async_endpoints` optional dependency extra is installed. For users installing from source, this can be done by running `uv pip install -e '.[async_endpoints]'` from the root directory of the NeMo Agent toolkit library. Similarly, for users installing from PyPI, this can be done by running `pip install 'nvidia-nat[async_endpoints]'`.
+The evaluation endpoint can be used to start evaluation jobs on a remote NeMo Agent toolkit server. This endpoint is only available when the `async_endpoints` optional dependency extra is installed. For users installing from source, this can be done by running `uv pip install -e '.[async_endpoints]'` from the root directory of the NeMo Agent toolkit library. Similarly, for users installing from PyPI, this can be done by running `pip install "nvidia-nat[async_endpoints]"`.
 
 ## Evaluation Endpoint Overview
 ```{mermaid}

docs/source/workflows/evaluate.md

@@ -34,7 +34,7 @@ uv pip install -e '.[profiling]'
 
 If you are installing from a package, you can install the sub-package by running the following command:
 ```bash
-uv pip install nvidia-nat[profiling]
+uv pip install "nvidia-nat[profiling]"
 ```
 
 ## Evaluating a Workflow

docs/source/workflows/llms/index.md

@@ -26,6 +26,7 @@ NVIDIA NeMo Agent toolkit supports the following LLM providers:
 | [OpenAI](https://openai.com) | `openai` | OpenAI API |
 | [AWS Bedrock](https://aws.amazon.com/bedrock/) | `aws_bedrock` | AWS Bedrock API |
 | [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/quickstart) | `azure_openai` | Azure OpenAI API |
+| [LiteLLM](https://github.com/BerriAI/litellm) | `litellm` | LiteLLM API |
 
 
 ## LLM Configuration
@@ -47,6 +48,9 @@ llms:
   azure_openai_llm:
     _type: azure_openai
     azure_deployment: gpt-4o-mini
+  litellm_llm:
+    _type: litellm
+    model_name: gpt-4o
 ```
 
 ### NVIDIA NIM
@@ -128,6 +132,22 @@ The Azure OpenAI LLM provider is defined by the {py:class}`~nat.llm.azure_openai
 `temperature` is model-gated and may not be supported by all models. See [Gated Fields](../../extend/gated-fields.md) for details.
 :::
 
+### LiteLLM
+
+LiteLLM is a general purpose LLM provider that can be used with any model provider that is supported by LiteLLM.
+See the [LiteLLM provider documentation](https://docs.litellm.ai/docs/providers) for more information on how to use LiteLLM.
+
+The LiteLLM LLM provider is defined by the {py:class}`~nat.llm.litellm_llm.LiteLlmModelConfig` class.
+
+* `model_name` - The name of the model to use (dependent on the model provider)
+* `api_key` - The API key to use for the model (dependent on the model provider)
+* `base_url` - The base URL to use for the model
+* `seed` - The seed to use for the model
+* `temperature` - The temperature to use for the model
+* `top_p` - The top-p value to use for the model
+* `max_retries` - The maximum number of retries for the request
+
+
 ## Testing Provider
 ### `nat_test_llm`
 `nat_test_llm` is a development and testing provider intended for examples and CI. It is not intended for production use.