Skip to content

Use valid package name in a test exception assert#10862

Closed
nineinchnick wants to merge 1 commit intotrinodb:masterfrom
nineinchnick:hive-test-exception-assert
Closed

Use valid package name in a test exception assert#10862
nineinchnick wants to merge 1 commit intotrinodb:masterfrom
nineinchnick:hive-test-exception-assert

Conversation

@nineinchnick
Copy link
Copy Markdown
Member

I'm seeing a lot of failures on master in a test that was recently introduced in #10613
with the following error:

tests               | 2022-01-31 12:13:37 INFO: FAILURE     /    io.trino.tests.product.hive.TestHiveCompatibility.testSmallDecimalFieldWrittenByOptimizedParquetWriterCannotBeReadByHive (Groups: storage_formats_detailed) took 0.7 seconds
tests               | 2022-01-31 12:13:37 SEVERE: Failure cause:
tests               | java.lang.AssertionError: 
tests               | Expecting message:
tests               |   <"java.io.IOException: parquet.io.ParquetDecodingException: Can not read value at 1 in block 0 in file hdfs://hadoop-master:9000/user/hive/warehouse/parquet_table_small_decimal_created_in_trino/20220131_062836_01082_j2ikr_8c200c2a-4add-4f35-a3ed-b70c4732e54b">
tests               | to match regex:
tests               |   <".* org.apache.parquet.io.ParquetDecodingException: Can not read value at 1 in block 0 in file .*">
tests               | but did not.

// Hive expects `FIXED_LEN_BYTE_ARRAY` for decimal values irrespective of the Parquet specification which allows `INT32`, `INT64` for short precision decimal types
assertQueryFailure(() -> onHive().executeQuery("SELECT a_decimal FROM " + tableName))
.hasMessageMatching(".* org.apache.parquet.io.ParquetDecodingException: Can not read value at 1 in block 0 in file .*");
.hasMessageMatching(".* parquet.io.ParquetDecodingException: Can not read value at 1 in block 0 in file .*");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should not validate the package at all? Why the package got changed? Maybe different Hadoop distros are using different parquet libraries?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original PR doesn't have the CI run completed, so I can't tell if it changed, or was just incorrect.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is the approach in #10847

@nineinchnick
Copy link
Copy Markdown
Member Author

Duplicate of #10847

@nineinchnick nineinchnick marked this as a duplicate of #10847 Jan 31, 2022
@findepi
Copy link
Copy Markdown
Member

findepi commented Jan 31, 2022

@nineinchnick @findinpath at some point Parquet had packages renamed, so different Hive versions have Parquet lib in either org.apache.parquet.io. or parquet.io..

@nineinchnick nineinchnick deleted the hive-test-exception-assert branch May 27, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants