feat(plugin-iceberg): Add support for MERGE INTO by acarpente-denodo · Pull Request #25470 · prestodb/presto

acarpente-denodo · 2025-07-01T16:27:49Z

Description

Iceberg connector support for the SQL MERGE INTO command.

Syntax:

MERGE INTO target_table [ [ AS ]  target_alias ]
USING { source_table | query } [ [ AS ] source_alias ]
ON search_condition
WHEN MATCHED THEN
    UPDATE SET ( column = expression [, ...] )
WHEN NOT MATCHED THEN
    INSERT [ column_list ]
    VALUES (expression, ...)

Example: MERGE INTO usage to update the sales information for existing products and insert the sales information for the new products in the market.

MERGE INTO product_sales AS s
    USING monthly_sales AS ms
    ON s.product_id = ms.product_id
WHEN MATCHED THEN
    UPDATE SET
        sales = sales + ms.sales
      , last_sale = ms.sale_date
      , current_price = ms.price
WHEN NOT MATCHED THEN
    INSERT (product_id, sales, last_sale, current_price)
    VALUES (ms.product_id, ms.sales, ms.sale_date, ms.price)

The Presto engine commit introduces an enum called RowChangeParadigm, which describes how a connector modifies rows. The iceberg connector will utilize the DELETE_ROW_AND_INSERT_ROW paradigm, as it represents an updated row as a combination of a deleted row followed by an inserted row. The CHANGE_ONLY_UPDATED_COLUMNS paradigm is meant for connectors that support updating individual columns of rows.

Note: Changes were made after reviewing the following Trino PR: trinodb/trino#7126
So, this PR is deeply inspired by Trino's implementation.

Motivation and Context

The MERGE INTO statement is commonly used to integrate data from two tables with different contents but similar structures.
For example, the source table could be part of a production transactional system, while the target table might be located in a data warehouse for analytics.
Regularly, MERGE operations are performed to update the analytics warehouse with the latest production data.
You can also use MERGE with tables that have different structures, as long as you can define a condition to match the rows between them.

Test Plan

Automated tests developed in IcebergDistributedTestBase.

Contributor checklist

Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
Documented new properties (with its default value), SQL syntax, functions, or other functionality.
If release notes are required, they follow the release notes guidelines.
Adequate tests were added if applicable.
CI passed.

Release Notes

== RELEASE NOTES ==

Iceberg Connector Changes
* Add support for MERGE command in the Iceberg connector.

aditi-pandit · 2025-07-07T21:15:32Z

@acarpente-denodo : Thanks for this contribution.

The code will need extensive changes for Presto C++ support. Have you investigated that ?

Did you try this code with a C++ worker (or side-car enabled ) ? We can do the full C++ support as a follow up, but its important the queries fail with a clean error message on Presto C++

acarpente-denodo · 2025-07-08T07:25:50Z

Hi @aditi-pandit,
Thanks for your feedback. No, I haven't investigated the support for Presto C++ yet. I also haven't tested this code with a C++ worker or with side-car enabled. At this point, I'm still focused on getting the MERGE INTO command fully working on Java. Once that's stable, I will plan for proper C++ support, including ensuring clean error messages when used with Presto C++.

yingsu00

Hi Adrian, were you able to build this PR? I got the following errors:

[ERROR] Failed to execute goal org.basepom.maven:duplicate-finder-maven-plugin:1.5.1:check (default) on project presto-main: Found duplicate classes/resources! -> [Help 1]

Also, how did you get the "handle not found" error? Were you running a query using the query runner? Were you able to get the correct plan? Would you please share the plan and full error stack?

yingsu00 · 2025-07-08T10:36:43Z

presto-docs/src/main/sphinx/sql.rst

    sql/grant
    sql/grant-roles
    sql/insert
+    sql/merge


At this commit, the merge into only supports parsing and not the full functionality. Let's move the documentation to the last commit when it's fully supported.

OK, it's done.

yingsu00 · 2025-07-08T10:37:14Z

presto-parser/src/test/java/com/facebook/presto/sql/parser/TestSqlParser.java

    }

+    @Test
+    public void testMergeInto()


Keep the original name testMerge()

OK, it's done.

yingsu00

Hi Adrian, were you able to build this PR? The CI build failed with these:

In file included from /__w/presto/presto/presto-native-execution/./presto_cpp/main/PeriodicHeartbeatManager.h:17:
/__w/presto/presto/presto-native-execution/./presto_cpp/presto_protocol/core/presto_protocol_core.h:1684:3: error: unknown type name 'com'
  com.facebook.presto.spi.MergeHandle handle = {};
  ^
/__w/presto/presto/presto-native-execution/./presto_cpp/presto_protocol/core/presto_protocol_core.h:1684:6: error: expected member name or ';' after declaration specifiers
  com.facebook.presto.spi.MergeHandle handle = {};
  ~~~^
/__w/presto/presto/presto-native-execution/./presto_cpp/presto_protocol/core/presto_protocol_core.h:1686:3: error: unknown type name 'ConnectorMergeTableHandle'; did you mean 'ConnectorDeleteTableHandle'?
  ConnectorMergeTableHandle connectorMergeTableHandle = {};
  ^~~~~~~~~~~~~~~~~~~~~~~~~
  ConnectorDeleteTableHandle
/__w/presto/presto/presto-native-execution/./presto_cpp/presto_protocol/core/presto_protocol_core.h:[313](https://github.com/prestodb/presto/actions/runs/16068848370/job/45348688297?pr=25470#step:17:314):8: note: 'ConnectorDeleteTableHandle' declared here
struct ConnectorDeleteTableHandle : public JsonEncodedSubclass {};
       ^
/__w/presto/presto/presto-native-execution/./presto_cpp/presto_protocol/core/presto_protocol_core.h:1709:3: error: unknown type name 'RowChangeParadigm'
  RowChangeParadigm paradigm = {};
  ^
4 errors generated.

Would you please fix the build errors first?

Also, how did you get the "handle not found" error? Were you running a query using the query runner? Since the build fails I wonder how you could do that. If you could run a query, were you able to get the correct plan? Would you please share the plan and full error stack? Thanks!

acarpente-denodo · 2025-07-15T07:38:57Z

Hi @yingsu00,

The PR is still in development. I'm still focused on getting the MERGE INTO command fully working on Java. The coordinator and workers are currently running on Java. I haven't investigated the MERGE INTO support for Presto C++ yet.

Automated tests have not been created yet. To test this enhancement, you need to perform a manual test. It is described at the end of the PR description in the section called "Test Plan".

I'm facing an error during the serialization of a new class called MergePartitioningHandle, which implements ConnectorPartitioningHandle interface

Caused by: java.lang.IllegalArgumentException: No connector for handle: MERGE [update = iceberg:INSTANCE]
at com.facebook.presto.metadata.HandleResolver.getId(HandleResolver.java:228)
at com.facebook.presto.metadata.HandleResolver.getId(HandleResolver.java:120)
at com.facebook.presto.metadata.AbstractTypedJacksonModule$InternalTypeResolver.idFromValueAndType(AbstractTypedJacksonModule.java:158)
at com.facebook.presto.metadata.AbstractTypedJacksonModule$InternalTypeResolver.idFromValue(AbstractTypedJacksonModule.java:150)
at com.fasterxml.jackson.databind.jsontype.impl.TypeSerializerBase.idFromValue(TypeSerializerBase.java:92)
at com.fasterxml.jackson.databind.jsontype.impl.TypeSerializerBase._generateTypeId(TypeSerializerBase.java:77)
at com.fasterxml.jackson.databind.jsontype.impl.TypeSerializerBase.writeTypePrefix(TypeSerializerBase.java:45)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:650)
at com.facebook.presto.metadata.AbstractTypedJacksonModule$InternalTypeSerializer.serialize(AbstractTypedJacksonModule.java:115)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
... 100 more

I think there is an issue with com.facebook.presto.iceberg.IcebergUpdateHandle#INSTANCE. You can find a complete description of this problem in the Slack thread I created a few weeks ago: https://prestodb.slack.com/archives/CLYMEEJ6S/p1749466521358289

The goal of this draft PR is to get assistance in fixing this problem.

steveburnett

Format nit. Thanks for the doc!

presto-docs/src/main/sphinx/sql/merge.rst

acarpente-denodo · 2025-08-12T22:01:19Z

Hi @steveburnett! Thank you for reviewing the feature documentation.

acarpente-denodo · 2025-11-26T19:37:34Z

@sourcery-ai

tdcmeehan · 2025-11-27T02:01:27Z

Can you please move the UI changes to a separate PR?

hantangwangd

Thanks for this great feature. I've left a few comments, mostly small things and a point for discussion.

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergPartitioningHandle.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergUpdateHandle.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMetadataColumn.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergUpdateablePageSource.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMetadataColumn.java

acarpente-denodo · 2025-12-17T17:58:13Z

Can you please move the UI changes to a separate PR?

Hi @tdcmeehan! OK. I moved the presto-ui changes to a separate PR: #26825

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMergeSink.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMetadataColumn.java

PingLiuPing

Thanks for the feature.

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergAbstractMetadata.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMergeSink.java

steveburnett · 2026-01-06T18:39:28Z

Is documentation no longer needed for this PR? Has the doc been moved to a different PR? Thanks!

acarpente-denodo · 2026-01-08T14:14:48Z

Is documentation no longer needed for this PR? Has the doc been moved to a different PR? Thanks!

Hi @steveburnett, the documentation for this feature was merged in this PR: Add SQL Support for MERGE INTO in Presto (engine)

So, no documentation is required for this PR.

PingLiuPing

Thanks, looks good overall. Could you consolidate the commit message little bit.
Leaving few comments on the tests.

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

PingLiuPing · 2026-01-08T14:31:01Z

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

+    {
+        String targetTable = "merge_inserts_" + randomTableSuffix();
+        assertUpdate(format("CREATE TABLE %s (customer VARCHAR, purchases INT, address VARCHAR)", targetTable));
+        assertUpdate(format("INSERT INTO %s (customer, purchases, address) VALUES ('Aaron', 11, 'Antioch'), ('Bill', 7, 'Buena')", targetTable), 2);


These code repeats lot, maybe adding a helper function.

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

PingLiuPing · 2026-01-08T14:35:57Z

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

+                (long) computeActual("SELECT count(*) FROM tpch.sf1.orders").getOnlyValue());
+
+        // verify copied rows: same total price
+        assertEquals(


For consistency, consider to use assertQuery . And all other places around.

acarpente-denodo · 2026-01-09T12:46:43Z

Hi @tdcmeehan, @steveburnett, @ethanyzhang and @PingLiuPing!

I've made the requested changes, and the PR has passed all automated tests. Could you please approve this PR?

PingLiuPing · 2026-01-12T10:47:13Z

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

+            assertUpdate("DROP TABLE " + targetTable);
+        }
+    }
+


Could you please add some tests for:

Empty source table

Empty target table

All rows matched

No rows matched

Hi @PingLiuPing! Sure, I created the new automated tests you requested.

PingLiuPing · 2026-01-12T10:50:19Z

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergMergeSink.java

+            fragments.addAll(writePositionDeletes(sink, deletion.rowsToDelete()));
+        });
+
+        return completedFuture(fragments);


nit: This method returns CompletableFuture but calls .join() synchronously, and this could blocking the thread and defeats the purpose of returning a CompletableFuture.

Hi @PingLiuPing! You are correct. I fixed it.

PingLiuPing

Thanks.

hantangwangd

@acarpente-denodo thanks for this feature, lgtm!

tdcmeehan

Thank you @acarpente-denodo

Support SQL MERGE in the Iceberg connector Cherry-pick of trinodb/trino@6cb188b Co-authored-by: David Phillips <david@acz.org>

SQL MERGE automated tests for Iceberg connector Cherry-pick of trinodb/trino@6cb188b Co-authored-by: David Phillips <david@acz.org>

- Improved CompletableFuture management. - Added new automted tests

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 3 times, most recently from 3e4437e to 5b65ae1 Compare July 4, 2025 07:59

yingsu00 reviewed Jul 9, 2025

View reviewed changes

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 3 times, most recently from ace4af6 to b02d82f Compare July 14, 2025 14:46

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch from 3465776 to 9658fd0 Compare July 21, 2025 10:50

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 6 times, most recently from 20d7c91 to 441ed5b Compare August 8, 2025 06:43

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 3 times, most recently from 09c9c54 to bca3929 Compare August 12, 2025 13:09

steveburnett requested changes Aug 12, 2025

View reviewed changes

presto-docs/src/main/sphinx/sql/merge.rst Outdated Show resolved Hide resolved

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch from 7bb8555 to e799d9f Compare August 13, 2025 09:32

acarpente-denodo requested a review from steveburnett August 13, 2025 10:42

steveburnett previously approved these changes Aug 13, 2025

View reviewed changes

acarpente-denodo dismissed steveburnett’s stale review via 9b22377 August 14, 2025 14:05

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 3 times, most recently from fd4bc78 to 8f49697 Compare August 18, 2025 18:22

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch from f9e99cb to 994f813 Compare November 26, 2025 17:08

tdcmeehan self-assigned this Nov 26, 2025

tdcmeehan requested a review from PingLiuPing November 26, 2025 19:38

hantangwangd reviewed Nov 29, 2025

View reviewed changes

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch from 994f813 to 6510337 Compare December 17, 2025 17:55

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 2 times, most recently from 07acbfe to 0b73712 Compare December 19, 2025 10:27

hantangwangd reviewed Dec 19, 2025

View reviewed changes

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch from 0b73712 to 298b708 Compare December 22, 2025 15:46

acarpente-denodo force-pushed the feature/20578_SQL_Support_for_MERGE_INTO branch 2 times, most recently from 79a5b6e to da4927d Compare December 29, 2025 20:15

PingLiuPing reviewed Dec 31, 2025

View reviewed changes

PingLiuPing reviewed Jan 8, 2026

View reviewed changes

PingLiuPing reviewed Jan 12, 2026

View reviewed changes

PingLiuPing approved these changes Jan 12, 2026

View reviewed changes

hantangwangd approved these changes Jan 12, 2026

View reviewed changes

tdcmeehan approved these changes Jan 12, 2026

View reviewed changes

acarpente-denodo and others added 3 commits January 14, 2026 09:14

Add SQL Support for MERGE INTO In Presto prestodb#20578 (iceberg)

97f48a2

Support SQL MERGE in the Iceberg connector Cherry-pick of trinodb/trino@6cb188b Co-authored-by: David Phillips <david@acz.org>

Add SQL Support for MERGE INTO In Presto prestodb#20578 (iceberg-tests)

dd9beda

SQL MERGE automated tests for Iceberg connector Cherry-pick of trinodb/trino@6cb188b Co-authored-by: David Phillips <david@acz.org>

Add SQL Support for MERGE INTO In Presto prestodb#20578 (iceberg)

92c97a0

- Improved CompletableFuture management. - Added new automted tests

This was referenced Mar 31, 2026

docs: Add release notes for 0.297 unix280/presto#51

Closed

docs: Add release notes for 0.297 unix280/presto#52

Open

prestodb-ci mentioned this pull request Apr 1, 2026

docs: Add release notes for 0.297 #27484

Open

15 tasks

Conversation

acarpente-denodo commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Test Plan

Contributor checklist

Release Notes

Uh oh!

aditi-pandit commented Jul 7, 2025

Uh oh!

acarpente-denodo commented Jul 8, 2025

Uh oh!

yingsu00 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yingsu00 left a comment

Choose a reason for hiding this comment

Uh oh!

acarpente-denodo commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

acarpente-denodo commented Aug 12, 2025

Uh oh!

acarpente-denodo commented Nov 26, 2025

Uh oh!

tdcmeehan commented Nov 27, 2025

Uh oh!

hantangwangd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

acarpente-denodo commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PingLiuPing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

steveburnett commented Jan 6, 2026

Uh oh!

acarpente-denodo commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PingLiuPing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

acarpente-denodo commented Jan 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

acarpente-denodo commented Jul 1, 2025 •

edited

Loading

acarpente-denodo commented Jul 15, 2025 •

edited

Loading

acarpente-denodo commented Dec 17, 2025 •

edited

Loading

acarpente-denodo commented Jan 8, 2026 •

edited

Loading