Prevent inserting nulls coming from expressions into non nullable columns

[homar]

Description

Is this change a fix, improvement, new feature, refactoring, or other?

a fix

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

core query engine

How would you describe this change to a non-technical end user or system administrator?

it prevents users from inserting nulls into non nullable columns

Related issues, pull requests, and links

fixes: #13434

Documentation

(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
(x) Release notes entries required with the following suggested text:

# Core
* Prevents inserting nulls that are results of expressions into non nullable columns

[findepi]

#13434

If the PR is supposed to fix (close) the issue, prepend the issue number with "fixes: ". Thanks

[findepi]

( #13551 )

// TODO (https://github.com/trinodb/trino/issues/13551) This doesn't fail for other connectors so
//  probably shouldn't fail for MariaDB either. Once fixed, remove test override.

(mind two spaces after // on the second comment line)

[findinpath]

reuse errorMessageForInsertIntoNotNullColumn("not_null_col")

[homar]

that won't work here

[findinpath]

There seems to be a problem here caused by the current implementation

I modified the test to run like this:

    public void testInsertIntoNotNullColumn()
    {
        try (TestTable table = new TestTable(getQueryRunner()::execute, "insert_not_null", "(nullable_col INTEGER, not_null_col INTEGER NOT NULL)")) {
            assertQuerySucceeds(format("INSERT INTO %s (nullable_col) SELECT nationkey FROM nation WHERE regionkey < 0", table.getName()));
        }

and I notice the following exception in the stacktrace:

java.lang.AssertionError: Expected query to succeed: INSERT INTO insert_not_null1evdhkhr9c (nullable_col) SELECT nationkey FROM nation WHERE regionkey < 0
...
    Suppressed: java.lang.Exception: SQL: INSERT INTO insert_not_null1evdhkhr9c (nullable_col) SELECT nationkey FROM nation WHERE regionkey < 0
.........
         Suppressed: java.lang.RuntimeException: Query: INSERT INTO `tpch`.`insert_not_null1evdhkhr9c` (`nullable_col`) SELECT `nullable_col` FROM `tpch`.`tmp_trino_2dc66901e1fa421f9cf3064225e9df13

There seems to be some side-effect of your changes.

[homar]

Sorry I don't understand your point here. This query fails because MariaDB does additional check that fails fast which is similar to what I did in LogicalPlanner here

[findinpath]

I get it now. The exception originates from MariaDB which complaints while copying the content from the temporary table towards the target table.

[homar]

Test failure not related and same as here #13556

[homar]

raptor failure is a know flaky #12385

[homar]

Second failure is io.trino.tests.product.iceberg.TestIcebergSparkCompatibility.testStatsAfterAddingPartitionField which is not a known flaky but root cause seems like a transient issue:
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/hive/warehouse/test_stats_after_adding_partition_field_co9knj4i0fkp/data/col1=4/00000-2016-70242f10-85f4-447a-ab96-8776b4233d2e-00001.parquet could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation.

[homar]

trino-hive failure seems not to be related - I opened new issue #13600

[homar]

In default, suite-7-non-generic TestIcebergSparkCompatibility.testTrinoReadsTrinoTableWithSparkDeletesAfterOptimizeAndCleanUp failed which is not a known issue but root cause seems to be a transient issue(like the one above - maybe there is some hidden problem somewhere?)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/hive/warehouse/test_spark_reads_trino_partitioned_table_with_deletes_after_expiring_snapshots_after_optimizeparquet-55279a4a1bd84f34b032479abbb27345/metadata/00003-fa053566-1af5-424f-af09-befb86799553.metadata.json could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation.

[findepi]

TestIcebergSparkCompatibility failed twice in a row (at different methods) with could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation..
We have some retries for that in Hive product tests.
Can this be applied in this class too?

[homar]

TestIcebergSparkCompatibility failed twice in a row (at different methods) with could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation.. We have some retries for that in Hive product tests. Can this be applied in this class too?

I think it can be. I will create a ticket for it

[homar]

TestIcebergSparkCompatibility failed twice in a row (at different methods) with could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation.. We have some retries for that in Hive product tests. Can this be applied in this class too?

#13601