Skip to content

Fix Hive sorted write failure when user identity contains @ sign (v2)#18131

Merged
findepi merged 3 commits intotrinodb:masterfrom
findepi:findepi/fix-hive-sorted-2
Jul 5, 2023
Merged

Fix Hive sorted write failure when user identity contains @ sign (v2)#18131
findepi merged 3 commits intotrinodb:masterfrom
findepi:findepi/fix-hive-sorted-2

Conversation

@findepi
Copy link
Copy Markdown
Member

@findepi findepi commented Jul 4, 2023

It's common for staging path to contain ${USER} placeholder and be on
local file system. When username contains an AT sign (name@domain),
the setSchemeToFileIfAbsent used to fail, because Location couldn't
parse e.g. file:///tmp/user@example.com, treating the @ sign as
userInfo separator. The parsing logic (split by @ first or split by /`
first) was different than in the URI class. This PR makes them more
similar.

Fixes #18132

findepi added 2 commits July 4, 2023 21:14
Locations look like URIs, so make more apparent similarities and
differences in how Location and URI work.
@findepi
Copy link
Copy Markdown
Member Author

findepi commented Jul 5, 2023

pt (hdp3, suite-3, ) failure doesn't look related

tests               | 2023-07-05 04:39:02 INFO: FAILURE     /    io.trino.tests.product.hive.TestHiveCoercionOnPartitionedTable.testHiveCoercionParquet (Groups: hive_coercion, jdbc) took 15.5 seconds
tests               | 2023-07-05 04:39:02 SEVERE: Failure cause:
tests               | java.lang.AssertionError: Could not find rows:
tests               | [null]
tests               | [null]
tests               | 
tests               | actual rows:
tests               | [2]
tests               | [2]
tests               | 	at io.trino.tests.product.hive.BaseTestHiveCoercion.assertQueryResults(BaseTestHiveCoercion.java:690)
tests               | 	at io.trino.tests.product.hive.BaseTestHiveCoercion.assertNestedSubFields(BaseTestHiveCoercion.java:560)
tests               | 	at io.trino.tests.product.hive.BaseTestHiveCoercion.doTestHiveCoercion(BaseTestHiveCoercion.java:145)
tests               | 	at io.trino.tests.product.hive.TestHiveCoercionOnPartitionedTable.testHiveCoercionParquet(TestHiveCoercionOnPartitionedTable.java:289)

will re-run.

edit: reported as #18134

It's common for staging path to contain `${USER}` placeholder and be on
local file system. When username contains an AT sign (`name@domain`),
the `setSchemeToFileIfAbsent` used to fail, because `Location` couldn't
parse e.g. `file:///tmp/user@example.com`, treating the `@` sign as
userInfo separator. The parsing logic (split by `@ first or split by `/`
first) was different than in the URI class. This commit makes them more
similar.
@findepi
Copy link
Copy Markdown
Member Author

findepi commented Jul 5, 2023

@findepi findepi force-pushed the findepi/fix-hive-sorted-2 branch from 504443c to c23752a Compare July 5, 2023 07:46
@findepi findepi merged commit 3a1a5c9 into trinodb:master Jul 5, 2023
@findepi findepi deleted the findepi/fix-hive-sorted-2 branch July 5, 2023 08:54
@github-actions github-actions bot added this to the 421 milestone Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed hive Hive connector

Development

Successfully merging this pull request may close these issues.

Hive sorted writes fail when user identity contains the @ AT sign

2 participants