fix(core): simplify the available function lists

[goldmedal]

Description

To avoid making the list too large for the LLM promotion, we only return the function name, description, and the function type.

Summary by CodeRabbit

• Refactor

  • Streamlined the function registration and metadata retrieval process to return only essential details (name, type, and an optional description), omitting previous parameter and return type specifics.
  • Adjusted the reported count of available functions for a clearer, more accurate overview.

• Tests

  • Updated test validations across multiple connectors to align with the simplified function metadata.

[coderabbitai]

Walkthrough

This pull request updates the test suite and core data retrieval logic. The constant DATAFUSION_FUNCTION_COUNT is decreased significantly, and multiple test cases across different connectors have been modified to expect None values for function metadata such as param_names, param_types, and return_type. In addition, the SQL query and conversion logic in the core context have been streamlined by introducing a new RemoteFunctionDto struct and an associated conversion to PyRemoteFunction. These changes also adjust the expected number of functions in the modeling core tests.

Changes

File(s)	Change Summary
ibis-server/tests/conftest.py	Updated constant DATAFUSION_FUNCTION_COUNT from 25941 → 276.
ibis-server/tests/routers/v3/connector/{bigquery, local_file, mysql, postgres, trino}/test_functions.py	Modified test cases for function lists: changed filter criteria (BigQuery) and updated expected function metadata values (param_names, param_types, return_type now set to None).
wren-core-py/src/context.rs	Revised get_regietered_functions: changed return type from Vec<RemoteFunction> to Vec<RemoteFunctionDto>, simplified SQL query, introduced new struct RemoteFunctionDto and added a From trait implementation for conversion to PyRemoteFunction.
wren-core-py/tests/test_modeling_core.py	Adjusted assertions: reduced expected function counts and changed expected metadata for functions (e.g., return types and parameter definitions now set to None).

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant Session as PySessionContext
    participant DB as Database
    Client->>Session: Call get_regietered_functions()
    Session->>DB: Execute simplified SQL query for distinct function metadata
    DB-->>Session: Return query results (rows)
    Session->>Session: Convert rows to RemoteFunctionDto objects
    Session->>Session: Transform RemoteFunctionDto to PyRemoteFunction (via From trait)
    Session->>Client: Return list of PyRemoteFunction objects

Loading

Possibly related PRs

• fix(ibis): fix the function white list of DuckDB #1129: Adjusts the constant used for function counts in coordination with the current significant reduction.
• chore: fix BigQuery v3 test cases #1031: Updates test assertions for the function list, aligning with the new expected behavior after the constant update.

Suggested labels

core, ibis, python

Suggested reviewers

• douenergy

Poem

I'm a bunny with a coder's zest,
Hopping through tests, I do my best.
Functions lightened like a breeze,
Metadata shifted with elegant ease.
With a twitching nose and a happy hop,
I celebrate these code drops non-stop!
🥕🐇 Happy coding!

---

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7e90148 and 29cd0dd.

📒 Files selected for processing (8)

• ibis-server/tests/conftest.py (1 hunks)
• ibis-server/tests/routers/v3/connector/bigquery/test_functions.py (1 hunks)
• ibis-server/tests/routers/v3/connector/local_file/test_functions.py (1 hunks)
• ibis-server/tests/routers/v3/connector/mysql/test_functions.py (1 hunks)
• ibis-server/tests/routers/v3/connector/postgres/test_functions.py (1 hunks)
• ibis-server/tests/routers/v3/connector/trino/test_functions.py (1 hunks)
• wren-core-py/src/context.rs (3 hunks)
• wren-core-py/tests/test_modeling_core.py (4 hunks)

🧰 Additional context used

🧬 Code Definitions (1)

wren-core-py/tests/test_modeling_core.py (3)

wren-core-py/src/context.rs (1)

• functions (92-95)

ibis-server/app/routers/v3/connector.py (1)

• functions (146-157)

wren-core/core/src/mdl/function.rs (3)

• return_type (224-226)
• return_type (287-289)
• param_types (46-49)

⏰ Context from checks skipped due to timeout of 90000ms (2)

• GitHub Check: ci
• GitHub Check: test

🔇 Additional comments (19)

ibis-server/tests/conftest.py (1)

14-14: Significant reduction in DATAFUSION_FUNCTION_COUNT

The constant has been reduced from 25941 to 276, which is a ~99% reduction. This aligns with the PR's objective to simplify function lists to prevent excessive size for LLM promotion.

ibis-server/tests/routers/v3/connector/postgres/test_functions.py (1)

65-66: Function metadata fields simplified

Setting param_types and return_type to None aligns with the PR's objective to simplify function lists by including only essential details (function name, description, and function type).

ibis-server/tests/routers/v3/connector/local_file/test_functions.py (1)

63-65: Function metadata simplified consistently

The changes to set param_names, param_types, and return_type to None are consistent with the simplification pattern applied across all connector test files, reducing the metadata to only essential details.

ibis-server/tests/routers/v3/connector/mysql/test_functions.py (1)

64-65: Consistent metadata simplification

The changes to set param_types and return_type to None maintain consistency with the other connector test files and support the PR's objective of streamlined function listings.

ibis-server/tests/routers/v3/connector/bigquery/test_functions.py (2)

51-52: Simplified function lookup logic.

The filter logic has been simplified to only check for function name "string_agg", removing the additional condition that was checking for specific parameter types.

---

59-61: Updated test expectations to align with simplified function lists.

These changes correctly reflect the PR's objective to streamline function metadata by having param_names, param_types, and return_type set to None. This simplification reduces the size of the function lists for LLM promotion.

ibis-server/tests/routers/v3/connector/trino/test_functions.py (1)

63-64: Updated test expectations for simplified function list structure.

These changes align with the PR objective to streamline function metadata by setting param_types and return_type to None. This is consistent with the pattern applied to other connector test files.

wren-core-py/tests/test_modeling_core.py (4)

109-109: Updated expected function count to reflect simplified function list.

This change reflects the significant reduction in the number of functions returned by the system due to the simplification of function lists.

---

121-121: Updated base function count when no remote function file is provided.

This change aligns with the new implementation that significantly reduces the function count from the previous implementation (reduced from 25941 to 276 based on the PR summary).

---

131-133: Updated test expectations for add_two function's metadata.

These changes correctly verify that the simplified function structure now returns None for previously detailed metadata fields (return_type, param_names, param_types), which aligns with the PR objective to reduce function list size.

---

145-147: Updated test expectations for add_custom function's metadata.

Similar to the changes for add_two, these assertions correctly verify that the new implementation returns None for the detailed metadata fields of add_custom.

wren-core-py/src/context.rs (8)

29-30: Added AsArray trait import.

Added the necessary import to support accessing array data in the updated implementation.

---

237-242: Updated method documentation to reflect simplified return values.

The documentation now clearly specifies that only the essential metadata fields (name, function_type, and description) are returned, which aligns with the PR's objective.

---

243-244: Changed return type to use the new simplified DTO structure.

The method signature has been updated to return Vec<RemoteFunctionDto> instead of Vec<RemoteFunction>, reflecting the shift to a simpler data structure with fewer fields.

---

246-252: Simplified SQL query to only retrieve essential function metadata.

The SQL query has been significantly streamlined to only select the name, function_type, and description, removing the complex CTE that was previously used to gather additional metadata. This is the core change that enables the reduction in data size.

---

264-266: Simplified array handling to match the reduced fields.

Updated the code to only retrieve the relevant columns from the result set, omitting the detailed metadata fields that are no longer needed.

---

272-277: Updated object creation to use the simplified data structure.

Now populating only the essential fields in the RemoteFunctionDto instead of the full RemoteFunction structure, which aligns with the PR objective to reduce data size.

---

283-287: Added lightweight DTO struct with only essential fields.

This new RemoteFunctionDto struct is a key part of the implementation, providing a minimal data structure that only contains the essential function metadata fields (name, function_type, description).

---

289-300: Added conversion from lightweight DTO to PyRemoteFunction.

This implementation properly converts the minimal RemoteFunctionDto to the full PyRemoteFunction structure, setting the excluded metadata fields to None. This ensures backward compatibility with existing code that expects the full structure.

✨ Finishing Touches

• 📝 Generate Docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai plan to trigger planning for file edits and PR creation.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[douenergy]

LGTM !
Thanks @goldmedal