[DNR] feat(native): Extend sidecar to support GPU execution

[pramodsatya]

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

• Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.
• If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* ... 
* ... 

Hive Connector Changes
* ... 
* ... 

If release note is NOT required, use:

== NO RELEASE NOTE ==

Summary by Sourcery

Add GPU/cuDF-aware planning and session configuration to native execution sidecar

New Features:

• Introduce cuDF-specific session properties initialized from CudfConfig and expose them via new sidecar HTTP endpoints, including catalog-specific property retrieval for native and cudf catalogs
• Integrate cuDF plan and expression support checks into Presto-to-Velox plan and expression conversion, failing fast when unsupported nodes or expressions are encountered

Enhancements:

• Wire cuDF plan/expression libraries into native build targets when PRESTO_ENABLE_CUDF is enabled and link new cuDF session properties library into the server
• Extend existing sidecar session properties endpoint structure to distinguish catalog-specific properties

Build:

• Update CMake targets to link against cuDF plan/exec libraries and to build the new cuDF session properties component when GPU support is enabled
• Comment out velox submodule sync/update commands in the native Makefile to avoid automatic submodule operations during build

Tests:

• Add unit tests validating cuDF session properties metadata, defaults, and initialization, and link cuDF-related libraries into existing native tests

[sourcery-ai]

Reviewer's Guide

Adds GPU (cuDF) execution awareness to the native sidecar by introducing cuDF-specific session properties, exposing them over new HTTP endpoints, and enforcing that converted Velox plans and expressions are compatible with cuDF when GPU support is enabled, along with build/test wiring for cuDF components.

Sequence diagram for Velox plan and expression conversion with cuDF validation

sequenceDiagram
  participant QC as QueryCoordinator
  participant VQPC as VeloxQueryPlanConverterBase
  participant VEC as VeloxExprConverter
  participant CPC as CudfPlanNodeChecker
  participant CEC as CudfExpressionChecker

  QC->>VQPC: toVeloxQueryPlan(prestoPlan)
  loop For each logical operator
    VQPC->>VQPC: build PlanNode
    alt ProjectNode
      VQPC->>CPC: isProjectNodeSupported(projectNode)
      alt Supported
        CPC-->>VQPC: true
        VQPC-->>QC: use projectNode
      else Not supported
        CPC-->>VQPC: false
        VQPC-->>QC: VELOX_FAIL("Project PlanNode not supported in cudf")
      end
    else FilterNode
      VQPC->>CPC: isFilterNodeSupported(filterNode)
      alt Supported
        CPC-->>VQPC: true
      else Not supported
        CPC-->>VQPC: false
        VQPC-->>QC: VELOX_FAIL("Filter PlanNode not supported in cudf")
      end
    else HashJoinNode
      VQPC->>CPC: isHashJoinNodeSupported(hashJoinNode)
      alt Supported
        CPC-->>VQPC: true
      else Not supported
        CPC-->>VQPC: false
        VQPC-->>QC: VELOX_FAIL("HashJoin PlanNode not supported in cudf")
      end
    else AggregationNode
      VQPC->>CPC: isAggregationNodeSupported(aggNode)
      alt Supported
        CPC-->>VQPC: true
      else Not supported
        CPC-->>VQPC: false
        VQPC-->>QC: VELOX_FAIL("Aggregation PlanNode not supported in cudf")
      end
    else TableScanNode
      VQPC->>CPC: isTableScanNodeSupported(scanNode)
      alt Supported
        CPC-->>VQPC: true
      else Not supported
        CPC-->>VQPC: false
        VQPC-->>QC: VELOX_FAIL("TableScan PlanNode not supported in cudf")
      end
    end
  end

  QC->>VEC: toVeloxExpr(rowExpression)
  VEC->>VEC: convert to TypedExprPtr
  VEC->>CEC: canBeEvaluatedByCudf([typedExpr])
  alt Expression supported by cuDF
    CEC-->>VEC: true
    VEC-->>QC: typedExpr
  else Expression not supported
    CEC-->>VEC: false
    VEC-->>QC: VELOX_FAIL("Expression not supported in cudf")
  end

Loading

Class diagram for cuDF session properties and configuration

classDiagram
  namespace facebook_presto {
    class CudfSessionProperties {
      +static const char* kCudfEnabled
      +static const char* kCudfDebugEnabled
      +static const char* kCudfMemoryResource
      +static const char* kCudfMemoryPercent
      +static const char* kCudfFunctionNamePrefix
      +static const char* kCudfAstExpressionEnabled
      +static const char* kCudfAstExpressionPriority
      +static const char* kCudfJitExpressionEnabled
      +static const char* kCudfJitExpressionPriority
      +static const char* kCudfAllowCpuFallback
      +static const char* kCudfLogFallback
      +static CudfSessionProperties* instance()
      +CudfSessionProperties()
      +json serialize() const
      -void addSessionProperty(name, description, type, isHidden, defaultValue)
      -unordered_map~string, shared_ptr~SessionProperty~~ sessionProperties_
    }

    class SessionProperty {
      +SessionProperty(name, description, typeString, isHidden, range, defaultValue)
      +protocol_SessionPropertyMetadata getMetadata() const
    }
  }

  namespace facebook_velox_cudf_velox {
    class CudfConfig {
      +static CudfConfig& getInstance()
      +bool enabled
      +bool debugEnabled
      +string memoryResource
      +int32_t memoryPercent
      +string functionNamePrefix
      +bool astExpressionEnabled
      +int32_t astExpressionPriority
      +bool jitExpressionEnabled
      +int32_t jitExpressionPriority
      +bool allowCpuFallback
      +bool logFallback
    }
  }

  namespace facebook_velox_type {
    class Type {
      +string toString() const
    }
  }

  CudfSessionProperties --> CudfConfig : reads defaults from
  CudfSessionProperties --> SessionProperty : creates
  CudfSessionProperties --> Type : uses in addSessionProperty

Loading

File-Level Changes

Change	Details	Files
Validate Velox query plans for cuDF compatibility when converting from Presto plans.	Include cuDF plan node checker headers when PRESTO_ENABLE_CUDF is defined. Wrap construction of ProjectNode, FilterNode, HashJoinNode, AggregationNode, TableScanNode, and related nodes with cuDF support checks. Fail query plan conversion with a clear error message if a constructed plan node type is not supported by cuDF.	presto-native-execution/presto_cpp/main/types/PrestoToVeloxQueryPlan.cpp
Validate converted expressions for cuDF GPU evaluation support.	Include cuDF expression checker when PRESTO_ENABLE_CUDF is defined. Refactor toVeloxExpr(protocol::RowExpression) to build a temporary TypedExprPtr and then, under PRESTO_ENABLE_CUDF, verify via cudf_velox::canBeEvaluatedByCudf before returning. Produce a detailed failure message and throw if the expression cannot be evaluated by cuDF while still throwing for unsupported RowExpression types.	presto-native-execution/presto_cpp/main/types/PrestoToVeloxExpr.cpp
Expose native and cuDF session properties via sidecar HTTP endpoints and wire cuDF session properties into the server binary.	Register existing /v1/properties/session endpoint as before and add a new catalog-specific endpoint /v1/properties/session/{catalog}. Route catalog==native to SessionProperties and catalog==cudf to new CudfSessionProperties, returning 404 for unsupported catalogs. Link presto_server_lib against the new presto_cudf_session_properties library in CMake.	presto-native-execution/presto_cpp/main/PrestoServer.cpp presto-native-execution/presto_cpp/main/CMakeLists.txt
Introduce cuDF-specific session properties backed by velox::cudf_velox::CudfConfig and expose them as Presto session properties.	Define CudfSessionProperties singleton with a registry of cuDF-related session properties (enablement, debug, memory resource/percent, function prefix, AST/JIT flags and priorities, CPU fallback and logging). Initialize each property using defaults from cudf_velox::CudfConfig and store as SessionProperty metadata (name, description, type, hidden flag, defaultValue). Provide JSON serialization of all cuDF session properties for exposure via HTTP endpoints.	presto-native-execution/presto_cpp/main/CudfSessionProperties.h presto-native-execution/presto_cpp/main/CudfSessionProperties.cpp
Wire cuDF plan/expression libraries into existing native components and tests, and add tests for cuDF session properties.	Link presto_velox_expr_conversion and presto_types against velox_cudf_plan when PRESTO_ENABLE_CUDF is set. Add new CudfSessionPropertiesTest to validate property initialization, default values against CudfConfig, and metadata (typeSignature, hidden). Link tests that rely on cuDF components (main tests and types tests) against velox_cudf_exec or velox_cudf_plan as appropriate. Comment out velox-submodule update commands in Makefile to avoid automatic submodule operations during build.	presto-native-execution/presto_cpp/main/types/CMakeLists.txt presto-native-execution/presto_cpp/main/tests/CMakeLists.txt presto-native-execution/presto_cpp/main/types/tests/CMakeLists.txt presto-native-execution/presto-native-execution/Makefile presto-native-execution/presto_cpp/main/tests/CudfSessionPropertiesTest.cpp

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.