Skip to content

feat(plugin-iceberg): Add support for tinyint and smallint datatypes by mapping them to Iceberg INTEGER type #27461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nishithakbhaskaran merged 1 commit into from
Apr 1, 2026
+187 −0
Merged

feat(plugin-iceberg): Add support for tinyint and smallint datatypes by mapping them to Iceberg INTEGER type #27461

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions presto-docs/src/main/sphinx/connector/iceberg.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2554,6 +2554,10 @@ Map of PrestoDB types to the relevant Iceberg types:
- ``BOOLEAN``
* - ``INTEGER``
- ``INTEGER``
* - ``TINYINT``
- ``INTEGER``
* - ``SMALLINT``
- ``INTEGER``
* - ``BIGINT``
- ``LONG``
* - ``REAL``
Expand Down
3 changes: 3 additions & 0 deletions presto-iceberg/src/main/java/com/facebook/presto/iceberg/TypeConverter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,9 @@ public static org.apache.iceberg.types.Type toIcebergType(Type type)
if (type instanceof IntegerType) {
return Types.IntegerType.get();
}
if (type.equals(SMALLINT) || type.equals(TINYINT)) {
return Types.IntegerType.get();
}
if (type instanceof BigintType) {
return Types.LongType.get();
}
Expand Down
180 changes: 180 additions & 0 deletions ...to-iceberg/src/test/java/com/facebook/presto/iceberg/TestIcebergSmallintTinyintTypes.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.facebook.presto.iceberg;

import com.facebook.presto.testing.MaterializedResult;
import com.facebook.presto.testing.QueryRunner;
import com.facebook.presto.tests.AbstractTestQueryFramework;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class TestIcebergSmallintTinyintTypes
extends AbstractTestQueryFramework
{
@Override
protected QueryRunner createQueryRunner()
throws Exception
{
return IcebergQueryRunner.builder().build().getQueryRunner();
}

@Test
public void testCreateTableWithSmallint()
{
String tableName = "test_smallint_table";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create table with SMALLINT column - should succeed with conversion to INTEGER
assertUpdate("CREATE TABLE " + tableName + " (id INTEGER, small_col SMALLINT)");
// Insert data
assertUpdate("INSERT INTO " + tableName + " VALUES (1, SMALLINT '100')", 1);
assertUpdate("INSERT INTO " + tableName + " VALUES (2, SMALLINT '32767')", 1); // Max SMALLINT value
assertUpdate("INSERT INTO " + tableName + " VALUES (3, SMALLINT '-32768')", 1); // Min SMALLINT value
// Verify data
MaterializedResult result = computeActual("SELECT * FROM " + tableName + " ORDER BY id");
assertEquals(result.getRowCount(), 3);
assertEquals(result.getMaterializedRows().get(0).getField(1), 100);
assertEquals(result.getMaterializedRows().get(1).getField(1), 32767);
assertEquals(result.getMaterializedRows().get(2).getField(1), -32768);
assertUpdate("DROP TABLE " + tableName);
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Extend partitioning test to cover TINYINT as a partition column or predicate target

Since this feature also adds TINYINT support, this test should exercise it directly. Either partition by tiny_col (or both small_col and tiny_col) and assert pruning via a TINYINT predicate, or at least add a predicate on tiny_col to the existing test so we verify partitioning behavior for both types.

Suggested implementation:

        String tableName = "test_partitioned_smallint_tinyint";
        assertUpdate("DROP TABLE IF EXISTS " + tableName);
        // Create partitioned table with SMALLINT and TINYINT partition columns
        assertUpdate("CREATE TABLE " + tableName + " (" +
                "id INTEGER, " +
                "small_col SMALLINT, " +
                "tiny_col TINYINT, " +
                "data VARCHAR) " +
                "WITH (PARTITIONING = ARRAY['small_col', 'tiny_col'])");

To fully implement the suggestion, the rest of testPartitionedTableWithSmallintTinyint should:

  1. Insert multiple rows that differ in both small_col and tiny_col so that pruning can be observed on each dimension.
  2. Include at least one query that filters only on tiny_col (e.g., WHERE tiny_col = <value>), and assert the expected row count and values.
  3. Optionally, add a query that filters on both small_col and tiny_col together to confirm partition pruning works when both partition columns are constrained.
  4. If the test currently only has predicates on small_col, extend or duplicate those assertions to use predicates on tiny_col analogously, ensuring both types are covered by partition pruning checks.

}

@Test
public void testCreateTableWithTinyint()
{
String tableName = "test_tinyint_table";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create table with TINYINT column - should succeed with conversion to INTEGER
assertUpdate("CREATE TABLE " + tableName + " (id INTEGER, tiny_col TINYINT)");
// Insert data
assertUpdate("INSERT INTO " + tableName + " VALUES (1, TINYINT '50')", 1);
assertUpdate("INSERT INTO " + tableName + " VALUES (2, TINYINT '127')", 1); // Max TINYINT value
assertUpdate("INSERT INTO " + tableName + " VALUES (3, TINYINT '-128')", 1); // Min TINYINT value
// Verify data
MaterializedResult result = computeActual("SELECT * FROM " + tableName + " ORDER BY id");
assertEquals(result.getRowCount(), 3);
assertEquals(result.getMaterializedRows().get(0).getField(1), 50);
assertEquals(result.getMaterializedRows().get(1).getField(1), 127);
assertEquals(result.getMaterializedRows().get(2).getField(1), -128);

assertUpdate("DROP TABLE " + tableName);
}

@Test
public void testCreateTableWithMixedIntegerTypes()
{
String tableName = "test_mixed_integer_types";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create table with all integer types
assertUpdate("CREATE TABLE " + tableName + " (" +
"id INTEGER, " +
"tiny_col TINYINT, " +
"small_col SMALLINT, " +
"int_col INTEGER, " +
"big_col BIGINT)");
// Insert data
assertUpdate("INSERT INTO " + tableName + " VALUES (1, TINYINT '10', SMALLINT '1000', 100000, BIGINT '10000000000')", 1);
// Verify data
MaterializedResult result = computeActual("SELECT * FROM " + tableName);
assertEquals(result.getRowCount(), 1);
assertEquals(result.getMaterializedRows().get(0).getField(0), 1);
assertEquals(result.getMaterializedRows().get(0).getField(1), 10);
assertEquals(result.getMaterializedRows().get(0).getField(2), 1000);
assertEquals(result.getMaterializedRows().get(0).getField(3), 100000);
assertEquals(result.getMaterializedRows().get(0).getField(4), 10000000000L);
assertUpdate("DROP TABLE " + tableName);
}

@Test
public void testInsertFromSelectWithSmallintTinyint()
{
String sourceTable = "test_source_table";
String targetTable = "test_target_table";
assertUpdate("DROP TABLE IF EXISTS " + sourceTable);
assertUpdate("DROP TABLE IF EXISTS " + targetTable);
// Create source table with SMALLINT and TINYINT
assertUpdate("CREATE TABLE " + sourceTable + " (id INTEGER, small_col SMALLINT, tiny_col TINYINT)");
assertUpdate("INSERT INTO " + sourceTable + " VALUES (1, SMALLINT '100', TINYINT '10')", 1);
assertUpdate("INSERT INTO " + sourceTable + " VALUES (2, SMALLINT '200', TINYINT '20')", 1);
// Create target table and insert from source
assertUpdate("CREATE TABLE " + targetTable + " (id INTEGER, small_col SMALLINT, tiny_col TINYINT)");
assertUpdate("INSERT INTO " + targetTable + " SELECT * FROM " + sourceTable, 2);
// Verify data
MaterializedResult result = computeActual("SELECT * FROM " + targetTable + " ORDER BY id");
assertEquals(result.getRowCount(), 2);
assertEquals(result.getMaterializedRows().get(0).getField(1), 100);
assertEquals(result.getMaterializedRows().get(0).getField(2), 10);
assertEquals(result.getMaterializedRows().get(1).getField(1), 200);
assertEquals(result.getMaterializedRows().get(1).getField(2), 20);
assertUpdate("DROP TABLE " + sourceTable);
assertUpdate("DROP TABLE " + targetTable);
}

@Test
public void testPartitionedTableWithSmallintTinyint()
{
String tableName = "test_partitioned_smallint_tinyint";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create partitioned table with SMALLINT partition column
assertUpdate("CREATE TABLE " + tableName + " (" +
"id INTEGER, " +
"small_col SMALLINT, " +
"tiny_col TINYINT, " +
"data VARCHAR) " +
"WITH (PARTITIONING = ARRAY['small_col'])");
// Insert data
assertUpdate("INSERT INTO " + tableName + " VALUES (1, SMALLINT '10', TINYINT '1', 'data1')", 1);
assertUpdate("INSERT INTO " + tableName + " VALUES (2, SMALLINT '20', TINYINT '2', 'data2')", 1);
assertUpdate("INSERT INTO " + tableName + " VALUES (3, SMALLINT '10', TINYINT '3', 'data3')", 1);
// Verify data with partition filter
MaterializedResult result = computeActual("SELECT * FROM " + tableName + " WHERE small_col = SMALLINT '10' ORDER BY id");
assertEquals(result.getRowCount(), 2);
assertEquals(result.getMaterializedRows().get(0).getField(0), 1);
assertEquals(result.getMaterializedRows().get(1).getField(0), 3);
assertUpdate("DROP TABLE " + tableName);
}

@Test
public void testCastFromSmallintTinyintToInteger()
{
String tableName = "test_cast_smallint_tinyint";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create table with SMALLINT and TINYINT
assertUpdate("CREATE TABLE " + tableName + " (small_col SMALLINT, tiny_col TINYINT)");
assertUpdate("INSERT INTO " + tableName + " VALUES (SMALLINT '100', TINYINT '10')", 1);
// Query with explicit cast to INTEGER
MaterializedResult result = computeActual("SELECT CAST(small_col AS INTEGER), CAST(tiny_col AS INTEGER) FROM " + tableName);
assertEquals(result.getRowCount(), 1);
assertEquals(result.getMaterializedRows().get(0).getField(0), 100);
assertEquals(result.getMaterializedRows().get(0).getField(1), 10);
assertUpdate("DROP TABLE " + tableName);
}

@Test
public void testArithmeticOperationsWithSmallintTinyint()
{
String tableName = "test_arithmetic_smallint_tinyint";
assertUpdate("DROP TABLE IF EXISTS " + tableName);
// Create table
assertUpdate("CREATE TABLE " + tableName + " (small_col SMALLINT, tiny_col TINYINT)");
assertUpdate("INSERT INTO " + tableName + " VALUES (SMALLINT '100', TINYINT '10')", 1);
// Test arithmetic operations
MaterializedResult result = computeActual("SELECT small_col + tiny_col, small_col - tiny_col, small_col * tiny_col FROM " + tableName);
assertEquals(result.getRowCount(), 1);
assertEquals(result.getMaterializedRows().get(0).getField(0), 110);
assertEquals(result.getMaterializedRows().get(0).getField(1), 90);
assertEquals(result.getMaterializedRows().get(0).getField(2), 1000);
assertUpdate("DROP TABLE " + tableName);
}
}
Loading