Add an integration test for the custom route front-end example #945
Conversation
Signed-off-by: David Gardner <[email protected]>
Signed-off-by: David Gardner <[email protected]>
…ests Signed-off-by: David Gardner <[email protected]>
Signed-off-by: David Gardner <[email protected]>
…to david-fe-simple-calc-custom-e2e Signed-off-by: David Gardner <[email protected]>
dagardner-nv
added
improvement
|
Warning Rate limit exceeded@dagardner-nv has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 18 minutes and 38 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughCentralizes test client creation by adding an async context manager Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Tester
participant Utils as nat.test.utils::build_nat_client
participant Worker as Worker (FastApiFrontEndPluginWorker)
participant App as FastAPI ASGI App
participant Lifespan as LifespanManager
participant Client as httpx.AsyncClient
Note over Tester,Utils: Test requests use centralized client builder
Tester->>Utils: call build_nat_client(config, worker_class?)
activate Utils
Utils->>Worker: instantiate worker (or provided class)
Worker-->>Utils: worker instance
Utils->>App: build_app(worker) → ASGI app
App-->>Utils: ASGI app ready
Utils->>Lifespan: LifespanManager(app) enter
Lifespan-->>Utils: lifespan active
Utils->>Client: create AsyncClient(ASGITransport(app))
Client-->>Utils: client ready
Utils-->>Tester: yield AsyncClient
deactivate Utils
Note over Tester,Client: Test performs request using yielded client
Tester->>Client: POST /get_request_metadata (headers, payload)
Client-->>Tester: 200 OK, JSON with route info
Note over Utils,Client: on context exit, client closed and lifespan stopped
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
examples/front_ends/simple_calculator_custom_routes/configs/config-metadata.yml(0 hunks)examples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.py(1 hunks)packages/nvidia_nat_test/src/nat/test/utils.py(2 hunks)tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.py(8 hunks)tests/nat/front_ends/fastapi/test_openai_compatibility.py(7 hunks)
💤 Files with no reviewable changes (1)
- examples/front_ends/simple_calculator_custom_routes/configs/config-metadata.yml
🧰 Additional context used
📓 Path-based instructions (9)
**/*.{py,yaml,yml}
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.
Files:
packages/nvidia_nat_test/src/nat/test/utils.pytests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pyexamples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
**/*.py
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).
**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial
Files:
packages/nvidia_nat_test/src/nat/test/utils.pytests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pyexamples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
packages/*/src/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Importable Python code inside packages must live under packages//src/
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
{src/**/*.py,packages/*/src/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
**/*
⚙️ CodeRabbit configuration file
**/*: # Code Review Instructions
- Ensure the code follows best practices and coding standards. - For Python code, follow
PEP 20 and
PEP 8 for style guidelines.- Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
Example:def my_function(param1: int, param2: str) -> bool: pass- For Python exception handling, ensure proper stack trace preservation:
- When re-raising exceptions: use bare
raisestatements to maintain the original stack trace,
and uselogger.error()(notlogger.exception()) to avoid duplicate stack trace output.- When catching and logging exceptions without re-raising: always use
logger.exception()
to capture the full stack trace information.Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any
words listed in the
ci/vale/styles/config/vocabularies/nat/reject.txtfile, words that might appear to be
spelling mistakes but are listed in theci/vale/styles/config/vocabularies/nat/accept.txtfile are OK.Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,
and should contain an Apache License 2.0 header comment at the top of each file.
- Confirm that copyright years are up-to date whenever a file is changed.
Files:
packages/nvidia_nat_test/src/nat/test/utils.pytests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pyexamples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
packages/**/*
⚙️ CodeRabbit configuration file
packages/**/*: - This directory contains optional plugin packages for the toolkit, each should contain apyproject.tomlfile. - Thepyproject.tomlfile should declare a dependency onnvidia-nator another package with a name starting
withnvidia-nat-. This dependency should be declared using~=<version>, and the version should be a two
digit version (ex:~=1.0).
- Not all packages contain Python code, if they do they should also contain their own set of tests, in a
tests/directory at the same level as thepyproject.tomlfile.
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
tests/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Unit tests reside under tests/ and should use markers defined in pyproject.toml (e.g., integration)
Files:
tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
⚙️ CodeRabbit configuration file
tests/**/*.py: - Ensure that tests are comprehensive, cover edge cases, and validate the functionality of the code. - Test functions should be named using thetest_prefix, using snake_case. - Any frequently repeated code should be extracted into pytest fixtures. - Pytest fixtures should define the name argument when applying the pytest.fixture decorator. The fixture
function being decorated should be named using thefixture_prefix, using snake_case. Example:
@pytest.fixture(name="my_fixture")
def fixture_my_fixture():
pass
Files:
tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
{tests/**/*.py,examples/*/tests/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
{tests/**/*.py,examples/*/tests/**/*.py}: Use pytest (with pytest-asyncio for async); name test files test_*.py; test functions start with test_; extract repeated code into fixtures; fixtures must set name in decorator and be named with fixture_ prefix
Mock external services with pytest_httpserver or unittest.mock; do not hit live endpoints
Mark expensive tests with @pytest.mark.slow or @pytest.mark.integration
Files:
tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.pytests/nat/front_ends/fastapi/test_openai_compatibility.py
examples/**/*
⚙️ CodeRabbit configuration file
examples/**/*: - This directory contains example code and usage scenarios for the toolkit, at a minimum an example should
contain a README.md or file README.ipynb.
- If an example contains Python code, it should be placed in a subdirectory named
src/and should
contain apyproject.tomlfile. Optionally, it might also contain scripts in ascripts/directory.- If an example contains YAML files, they should be placed in a subdirectory named
configs/. - If an example contains sample data files, they should be placed in a subdirectory nameddata/, and should
be checked into git-lfs.
Files:
examples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.py
🧬 Code graph analysis (4)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
src/nat/front_ends/fastapi/fastapi_front_end_plugin_worker.py (2)
FastApiFrontEndPluginWorker(222-1253)build_app(129-154)
tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.py (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
build_nat_client(92-108)
examples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.py (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
build_nat_client(92-108)
tests/nat/front_ends/fastapi/test_openai_compatibility.py (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
build_nat_client(92-108)
🪛 Ruff (0.13.3)
examples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.py
32-32: Unused function argument: restore_environ
(ARG001)
🔇 Additional comments (4)
tests/nat/front_ends/fastapi/test_fastapi_front_end_plugin.py (1)
36-36: LGTM!The refactor successfully centralizes the client-building logic by using the new
build_nat_clienthelper fromnat.test.utils. All usages correctly replace the previous_build_clientcalls, and the optionalworker_classparameter is properly passed where needed.Also applies to: 69-69, 114-114, 160-160, 183-183, 212-212, 269-269, 291-291
tests/nat/front_ends/fastapi/test_openai_compatibility.py (1)
31-31: LGTM!The refactor correctly replaces all
_build_clientcalls with the centralizedbuild_nat_clienthelper. The changes maintain the same behavior while eliminating code duplication.Also applies to: 185-185, 260-260, 298-298, 373-373, 441-441, 517-517
examples/front_ends/simple_calculator_custom_routes/tests/test_simple_calculator_custom_routes.py (2)
31-35: Static analysis warning is a false positive.The
restore_environparameter is intentionally unused. This is a common pytest pattern where a fixture depends on another fixture purely for its side effect (in this case, restoring the environment after the test). The parameter establishes the dependency chain but doesn't need to be used in the function body.
38-56: LGTM!The integration test is well-structured:
- Properly marked with
@pytest.mark.integrationand requiresnvidia_api_key- Loads the config and builds a test client using the new centralized helper
- Validates the response status and structure
- Includes helpful assertion messages for debugging
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: David Gardner <[email protected]>
Signed-off-by: David Gardner <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
1-27: Add a module-level docstring.This module contains public utility functions for testing but lacks a module-level docstring. Per the coding guidelines, every public module must have a Google-style docstring.
Add a module-level docstring after the license header:
# limitations under the License. + +""" +Utility functions for NAT testing. + +This module provides helper functions for locating example files, running workflows, +and building test clients for integration testing. +""" import importlib.resources
🧹 Nitpick comments (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
91-96: Consider consolidating TYPE_CHECKING imports.The
TYPE_CHECKINGconstant is imported twice: once at line 23 (typing.TYPE_CHECKING) and again at line 91 (from typing import TYPE_CHECKING). While functional, consolidating all TYPE_CHECKING blocks at the top of the file would improve consistency.If you prefer to keep type hints near their usage, use the already-imported
typing.TYPE_CHECKINGreference:-from typing import TYPE_CHECKING - -if TYPE_CHECKING: +if typing.TYPE_CHECKING: from collections.abc import AsyncIterator from httpx import AsyncClient
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/nvidia_nat_test/src/nat/test/utils.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{py,yaml,yml}
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
**/*.py
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).
**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
packages/*/src/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Importable Python code inside packages must live under packages//src/
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
{src/**/*.py,packages/*/src/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
**/*
⚙️ CodeRabbit configuration file
**/*: # Code Review Instructions
- Ensure the code follows best practices and coding standards. - For Python code, follow
PEP 20 and
PEP 8 for style guidelines.- Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
Example:def my_function(param1: int, param2: str) -> bool: pass- For Python exception handling, ensure proper stack trace preservation:
- When re-raising exceptions: use bare
raisestatements to maintain the original stack trace,
and uselogger.error()(notlogger.exception()) to avoid duplicate stack trace output.- When catching and logging exceptions without re-raising: always use
logger.exception()
to capture the full stack trace information.Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any
words listed in the
ci/vale/styles/config/vocabularies/nat/reject.txtfile, words that might appear to be
spelling mistakes but are listed in theci/vale/styles/config/vocabularies/nat/accept.txtfile are OK.Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,
and should contain an Apache License 2.0 header comment at the top of each file.
- Confirm that copyright years are up-to date whenever a file is changed.
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
packages/**/*
⚙️ CodeRabbit configuration file
packages/**/*: - This directory contains optional plugin packages for the toolkit, each should contain apyproject.tomlfile. - Thepyproject.tomlfile should declare a dependency onnvidia-nator another package with a name starting
withnvidia-nat-. This dependency should be declared using~=<version>, and the version should be a two
digit version (ex:~=1.0).
- Not all packages contain Python code, if they do they should also contain their own set of tests, in a
tests/directory at the same level as thepyproject.tomlfile.
Files:
packages/nvidia_nat_test/src/nat/test/utils.py
🧬 Code graph analysis (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
src/nat/front_ends/fastapi/fastapi_front_end_plugin_worker.py (1)
build_app(129-154)
🔇 Additional comments (1)
packages/nvidia_nat_test/src/nat/test/utils.py (1)
97-132: LGTM! Well-implemented public API.The
build_nat_clientfunction is correctly implemented with:
- Proper async context manager pattern using
@asynccontextmanager- Complete type hints including
AsyncIterator[AsyncClient]return type- Comprehensive Google-style docstring
- Sound implementation using
LifespanManagerfor app lifecycle andASGITransportfor test client- Appropriate default handling for
worker_classThis addresses the previous review feedback requesting type hints and documentation.
Signed-off-by: David Gardner <[email protected]>
|
/merge |
2 participants
Description
examples/front_ends/simple_calculator_custom_routesconfig-metadata.yml_build_clienthelper method tobuild_nat_clientin the tests package to be shared among testsBy Submitting this PR I confirm:
Summary by CodeRabbit
New Features
Tests
Chores