Skip to content

[pos] Fix connector caused test failures#26068

Merged
tanjialiang merged 1 commit intoprestodb:masterfrom
tanjialiang:connector_test
Sep 19, 2025
Merged

[pos] Fix connector caused test failures#26068
tanjialiang merged 1 commit intoprestodb:masterfrom
tanjialiang:connector_test

Conversation

@tanjialiang
Copy link
Contributor

@tanjialiang tanjialiang commented Sep 17, 2025

Description

Fixes 3 tests that were previously failing due to connector issues. Connectors for presto on spark native were not created correctly. Because presto on spark native query runner does not allow adding new catalogs after creation, the fix allows presto on spark native to be able to create query runners with multiple catalogs up-front.

== NO RELEASE NOTE ==

@tanjialiang tanjialiang requested review from a team as code owners September 17, 2025 22:31
@prestodb-ci prestodb-ci added the from:Meta PR from Meta label Sep 17, 2025
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Sep 17, 2025

Reviewer's Guide

PR fixes connector-induced test failures by enabling Presto Spark native runners to register multiple catalogs at creation time. It introduces an overloaded createHiveRunner that builds native- and Java-side catalogs up-front, corrects the default catalog mapping for Tpch runners, and updates test classes to leverage the new utility and clean up obsolete overrides.

File-Level Changes

Change Details Files
Allow creation of native query runners with multiple catalogs up-front
  • Add overloaded createHiveRunner method accepting additionalCatalogs
  • Build native-side connectors map including default and additional catalogs
  • Invoke queryRunner.createCatalog for each additional catalog on Java side
  • Retain JSON function namespace setup
presto-native-execution/src/test/java/com/facebook/presto/spark/PrestoSparkNativeQueryRunnerUtils.java
Correct default catalog mapping in Tpch runner
  • Change mapping key to 'tpchstandard'
  • Ensure connector.name property set to 'tpch'
presto-native-execution/src/test/java/com/facebook/presto/spark/PrestoSparkNativeQueryRunnerUtils.java
Update general queries test to register additional catalog
  • Import ImmutableMap
  • Call createHiveRunner with hivecached catalog in TestPrestoSparkNativeGeneralQueries
presto-native-execution/src/test/java/com/facebook/presto/spark/TestPrestoSparkNativeGeneralQueries.java
Clean up Tpch connector tests and remove obsolete overrides
  • Remove ignored test overrides no longer needed
  • Add import for ImmutableMap
presto-native-execution/src/test/java/com/facebook/presto/spark/TestPrestoSparkNativeTpchConnectorQueries.java

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `presto-native-execution/src/test/java/com/facebook/presto/spark/TestPrestoSparkNativeGeneralQueries.java:115-32` </location>
<code_context>
     @Ignore
     public void testArrayAndMapFunctions() {}

-    // VeloxRuntimeError: it != connectors().end() Connector with ID 'hivecached' not registered
-    @Override
-    @Ignore
-    public void testCatalogWithCacheEnabled() {}
</code_context>

<issue_to_address>
**suggestion (testing):** Restoring previously ignored tests could improve coverage.

Since the runner now supports multiple catalogs, please re-enable 'testCatalogWithCacheEnabled' and confirm its assertions to ensure the recent changes are properly tested.

Suggested implementation:

```java
    @Override
    public void testCatalogWithCacheEnabled() {
        // Ensure the runner supports multiple catalogs and cache-enabled catalog works
        PrestoSparkNativeQueryRunner runner = PrestoSparkNativeQueryRunnerUtils.createHiveRunner(
            ImmutableMap.of(
                "hivecached", ImmutableMap.of("connector.name", "hive"),
                "hive", ImmutableMap.of("connector.name", "hive")
            )
        );
        // Example assertion: query both catalogs and check results
        assertQuery(runner, "SELECT * FROM hivecached.default.nation", "SELECT * FROM hive.default.nation");
        // Add more assertions as needed to confirm cache behavior
    }

```

If the `assertQuery` method or the runner setup requires additional configuration for cache validation, you may need to add more specific assertions or setup steps. Also, ensure that the test data exists in both catalogs for the assertion to pass.
</issue_to_address>

### Comment 2
<location> `presto-native-execution/src/test/java/com/facebook/presto/spark/TestPrestoSparkNativeTpchConnectorQueries.java:43-44` </location>
<code_context>
     @Override
     protected QueryRunner createQueryRunner()
     {
</code_context>

<issue_to_address>
**suggestion (testing):** Consider re-enabling previously ignored tests for 'tpchstandard' catalog.

Previously, 'testTpchTinyTables' and 'testTpchDateFilter' were ignored due to connector registration errors. With the updated runner creation, these tests may now work. Please re-enable and verify that the original issues are resolved.

Suggested implementation:

```java

```

```java
public void testTpchTinyTables()
{
    // test implementation
}

public void testTpchDateFilter()
{
    // test implementation
}

```

If the test methods are currently commented out or missing, you will need to restore their full implementations. Also, after re-enabling, run the tests to verify that the original connector registration errors are resolved.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@@ -31,7 +32,9 @@ public class TestPrestoSparkNativeGeneralQueries
@Override
protected QueryRunner createQueryRunner()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Restoring previously ignored tests could improve coverage.

Since the runner now supports multiple catalogs, please re-enable 'testCatalogWithCacheEnabled' and confirm its assertions to ensure the recent changes are properly tested.

Suggested implementation:

    @Override
    public void testCatalogWithCacheEnabled() {
        // Ensure the runner supports multiple catalogs and cache-enabled catalog works
        PrestoSparkNativeQueryRunner runner = PrestoSparkNativeQueryRunnerUtils.createHiveRunner(
            ImmutableMap.of(
                "hivecached", ImmutableMap.of("connector.name", "hive"),
                "hive", ImmutableMap.of("connector.name", "hive")
            )
        );
        // Example assertion: query both catalogs and check results
        assertQuery(runner, "SELECT * FROM hivecached.default.nation", "SELECT * FROM hive.default.nation");
        // Add more assertions as needed to confirm cache behavior
    }

If the assertQuery method or the runner setup requires additional configuration for cache validation, you may need to add more specific assertions or setup steps. Also, ensure that the test data exists in both catalogs for the assertion to pass.

xiaoxmeng
xiaoxmeng previously approved these changes Sep 17, 2025
singcha
singcha previously approved these changes Sep 17, 2025
Copy link
Contributor

@singcha singcha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tanjialiang tanjialiang dismissed stale reviews from singcha and xiaoxmeng via 9bf6ed4 September 18, 2025 19:37
@tanjialiang tanjialiang force-pushed the connector_test branch 3 times, most recently from a481fdd to 820e6e9 Compare September 18, 2025 20:22
@tanjialiang tanjialiang merged commit 105af44 into prestodb:master Sep 19, 2025
107 of 108 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:Meta PR from Meta

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants