fix: TTC must await get_function from builder

[willkill07]

Description

This fixes a coroutine missing await when running with TTC.

It also fixes the related type hints for input+output schema.

Closes nvbugs-5560791

By Submitting this PR I confirm:

• I am familiar with the Contributing Guidelines.
• We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  • Any contribution which contains commits that are not Signed-Off will not be accepted.
• When the PR is ready for review, new or existing tests cover these changes.
• When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

• New Features

  • None

• Bug Fixes

  • Improved reliability by ensuring function resolution is awaited before execution, reducing timing-related issues.

• Refactor

  • Transitioned a function retrieval step to asynchronous execution for smoother control flow.
  • Updated schema handling to use type-based definitions, improving compatibility with dynamic structured outputs.

• Chores

  • Internal typing adjustments to align with expected schema types.

[coderabbitai]

Walkthrough

Converted function retrieval and update calls to async/await in execute_score_select_function.py. Adjusted local typing in ttc_tool_wrapper_function.py from BaseModel instances to BaseModel subclass types (and optional None for output). No exported/public API changes reported.

Changes

Cohort / File(s)	Summary
Async function handling src/nat/experimental/test_time_compute/functions/execute_score_select_function.py	Await builder.get_function(...) and await executable_fn.update(...), making function retrieval and update asynchronous; control flow now pauses until function resolution completes.
Schema typing adjustments src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py	Change local types: fn_input_schema: type[BaseModel], fn_output_schema: type[BaseModel] | type[None]; no logic change.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Caller
  participant Builder
  participant ExecutableFn

  Note over Caller: Execute score/select (async flow)
  Caller->>Builder: get_function(config, ...)
  activate Builder
  Builder-->>Caller: Promise<ExecutableFn>
  deactivate Builder

  Caller->>Caller: await get_function(...)
  Caller->>ExecutableFn: update(context, inputs)
  activate ExecutableFn
  ExecutableFn-->>Caller: Promise<UpdateResult>
  deactivate ExecutableFn

  Caller->>Caller: await update(...)
  Note over Caller: Proceed after awaited results

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title concisely summarizes the main change—making TTC await the builder’s get_function—remains well under the 72-character limit, and clearly reflects the bug fix as intended.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py (1)

86-90: Remove duplicate conditional check.

Lines 86-90 contain duplicate logic—the same condition if not augmented_function.has_single_output: is checked twice in succession. Remove the duplicate to improve code clarity.

Apply this diff to remove the duplicate:

     if not augmented_function.has_single_output:
         raise ValueError("TTCToolWrapperFunction only supports functions with a single output.")

-    if not augmented_function.has_single_output:
-        raise ValueError("TTCToolWrapperFunction only supports functions with a single output.")
-
     if augmented_function.description and augmented_function.description != "":

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 14cfda4 and 03e5b95.

📒 Files selected for processing (2)

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py (1 hunks)
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (6)

**/*.{py,yaml,yml}

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

**/*.py

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).

**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

src/**/*.py

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

All importable Python code must live under src/ (or packages//src/)

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

src/nat/**/*

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

Changes in src/nat should prioritize backward compatibility

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

---

⚙️ CodeRabbit configuration file

This directory contains the core functionality of the toolkit. Changes should prioritize backward compatibility.

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

{src/**/*.py,packages/*/src/**/*.py}

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

**/*

⚙️ CodeRabbit configuration file

**/*: # Code Review Instructions

• Ensure the code follows best practices and coding standards. - For Python code, follow
  PEP 20 and
  PEP 8 for style guidelines.
• Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
  Example:

  def my_function(param1: int, param2: str) -> bool:
      pass

• For Python exception handling, ensure proper stack trace preservation:
  • When re-raising exceptions: use bare raise statements to maintain the original stack trace,
    and use logger.error() (not logger.exception()) to avoid duplicate stack trace output.
  • When catching and logging exceptions without re-raising: always use logger.exception()
    to capture the full stack trace information.

Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any

words listed in the ci/vale/styles/config/vocabularies/nat/reject.txt file, words that might appear to be
spelling mistakes but are listed in the ci/vale/styles/config/vocabularies/nat/accept.txt file are OK.

Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,

and should contain an Apache License 2.0 header comment at the top of each file.

• Confirm that copyright years are up-to date whenever a file is changed.

Files:

• src/nat/experimental/test_time_compute/functions/execute_score_select_function.py
• src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py

🧬 Code graph analysis (2)

src/nat/experimental/test_time_compute/functions/execute_score_select_function.py (1)

examples/object_store/user_report/tests/test_objext_store_example_user_report_tool.py (1)

• builder (29-40)

src/nat/experimental/test_time_compute/functions/ttc_tool_wrapper_function.py (1)

packages/nvidia_nat_mcp/src/nat/plugins/mcp/client_base.py (1)

• input_schema (612-616)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: CI Pipeline / Check

🔇 Additional comments (1)

src/nat/experimental/test_time_compute/functions/execute_score_select_function.py (1)

49-49: LGTM! Critical async fix applied correctly.

The await keyword is correctly added to resolve the coroutine from builder.get_function(). This ensures executable_fn is properly initialized before use throughout the function.

[willkill07]

/merge