Skip to content

Suppress single table errors during metadata listing #21981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 16, 2024
Merged

Suppress single table errors during metadata listing #21981

merged 3 commits into from
May 16, 2024

Conversation

@pajaks
Copy link
Member

@pajaks pajaks commented May 15, 2024 •
edited
Loading

Description

In case of table error (e.g Table SerdeInfo is null) listing is failing.
This change filter outs views causing errors from lists returned when listing tables.

Additional context and related issues

Example stack trace:

  "type": "io.trino.spi.TrinoException",
   "message": "Table SerdeInfo is null for table 'table'
   "stack":
     "io.trino.plugin.hive.metastore.glue.converter.GlueToTrinoConverter.convertTable(GlueToTrinoConverter.java:167)",
     "io.trino.plugin.hive.metastore.glue.GlueHiveMetastore.getTable(GlueHiveMetastore.java:282)",
     "io.trino.plugin.hive.metastore.tracing.TracingHiveMetastore.lambda$getTable$2(TracingHiveMetastore.java:103)",
     "io.trino.plugin.hive.metastore.tracing.Tracing.withTracing(Tracing.java:39)",
     "io.trino.plugin.hive.metastore.tracing.TracingHiveMetastore.getTable(TracingHiveMetastore.java:103)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.loadTable(CachingHiveMetastore.java:438)",
     "com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:169)",
     "com.google.common.cache.CacheLoader$1.load(CacheLoader.java:192)",
     "io.trino.cache.EvictableCache$TokenCacheLoader.load(EvictableCache.java:447)",
     "io.trino.cache.EvictableCache$TokenCacheLoader.load(EvictableCache.java:433)",
     "com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3576)",
     "com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318)",
     "com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2191)",
     "com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2081)",
     "com.google.common.cache.LocalCache.get(LocalCache.java:4019)",
     "com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4042)",
     "com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5024)",
     "io.trino.cache.EvictableCache.get(EvictableCache.java:145)",
     "com.google.common.cache.AbstractLoadingCache.getUnchecked(AbstractLoadingCache.java:53)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.getOptional(CachingHiveMetastore.java:288)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.getTable(CachingHiveMetastore.java:433)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.loadTable(CachingHiveMetastore.java:438)",
     "com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:169)",
     "io.trino.cache.EvictableCache$TokenCacheLoader.load(EvictableCache.java:447)",
     "io.trino.cache.EvictableCache$TokenCacheLoader.load(EvictableCache.java:433)",
     "com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3576)",
     "com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318)",
     "com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2191)",
     "com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2081)",
     "com.google.common.cache.LocalCache.get(LocalCache.java:4019)",
     "com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4042)",
     "com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5024)",
     "io.trino.cache.EvictableCache.get(EvictableCache.java:145)",
     "com.google.common.cache.AbstractLoadingCache.getUnchecked(AbstractLoadingCache.java:53)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.getOptional(CachingHiveMetastore.java:288)",
     "io.trino.plugin.hive.metastore.cache.CachingHiveMetastore.getTable(CachingHiveMetastore.java:433)",
     "io.trino.plugin.hive.HiveMetastoreClosure.getTable(HiveMetastoreClosure.java:91)",
     "io.trino.plugin.hive.metastore.SemiTransactionalHiveMetastore.getTable(SemiTransactionalHiveMetastore.java:263)",
     "io.trino.plugin.hive.HiveMetadata.getView(HiveMetadata.java:2888)",
     "io.trino.tracing.TracingConnectorMetadata.getView(TracingConnectorMetadata.java:878)",
     "io.trino.metadata.MetadataManager.getViewInternal(MetadataManager.java:1545)",
     "io.trino.metadata.MetadataManager.isView(MetadataManager.java:1477)",
     "io.trino.metadata.MetadataManager.getRelationTypeIfExists(MetadataManager.java:582)",
     "io.trino.metadata.MetadataManager.listTables(MetadataManager.java:514)",
     "io.trino.tracing.TracingMetadata.listTables(TracingMetadata.java:319)",
     "io.trino.metadata.MetadataListing.doListTables(MetadataListing.java:128)",
     "io.trino.metadata.MetadataListing.listTables(MetadataListing.java:119)",
     "io.trino.connector.informationschema.InformationSchemaPageSource.addTablesRecords(InformationSchemaPageSource.java:292)",
     "io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:221)",
     "io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:185)",
     "io.trino.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:384)",
     "io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:412)",
     "io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)",
       ...

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Hive
* Fix failure when listing Hive tables having unsupported syntax. ({issue}`21981`)

@cla-bot cla-bot bot added the cla-signed label May 15, 2024
@github-actions github-actions bot added the hive Hive connector label May 15, 2024
@pajaks pajaks requested review from ebyhr and findepi and removed request for ebyhr May 15, 2024 13:00
@pajaks pajaks force-pushed the pajaks/listing_tables_skip_errors branch from 196af15 to 7ff0098 Compare May 15, 2024 13:05
@pajaks pajaks requested a review from findinpath May 15, 2024 13:50
ebyhr
ebyhr reviewed May 16, 2024
View reviewed changes
@findinpath
Copy link
Contributor

findinpath commented May 16, 2024

In case of table error (e.g Table SerdeInfo is null) listing is failing.

Pls add the stacktrace of the exception

@pajaks pajaks force-pushed the pajaks/listing_tables_skip_errors branch 2 times, most recently from 35b94c7 to a44a0d9 Compare May 16, 2024 08:13
@pajaks
Copy link
Member Author

pajaks commented May 16, 2024

@ebyhr @findepi Could you run with secrets?

@ebyhr
Copy link
Member

ebyhr commented May 16, 2024

/test-with-secrets sha=a44a0d916f79340a97756ae714f8342fa8f01630

findepi
findepi approved these changes May 16, 2024
View reviewed changes
@github-actions
Copy link

github-actions bot commented May 16, 2024 •
edited
Loading

The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/9109152980

@pajaks
Copy link
Member Author

pajaks commented May 16, 2024

Tests with secrets failed on plugin/trino-redis with known issue: #18489

@pajaks pajaks force-pushed the pajaks/listing_tables_skip_errors branch from a44a0d9 to 477bedb Compare May 16, 2024 10:56
@pajaks
Copy link
Member Author

pajaks commented May 16, 2024

CI hit: #21862

@pajaks pajaks requested a review from ebyhr May 16, 2024 12:58
@findepi findepi merged commit 429f8a8 into trinodb:master May 16, 2024
@github-actions github-actions bot added this to the 449 milestone May 16, 2024
@colebow colebow mentioned this pull request May 29, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@findepi findepi findepi approved these changes

@findinpath findinpath Awaiting requested review from findinpath

@ebyhr ebyhr Awaiting requested review from ebyhr

Assignees

No one assigned

Labels

cla-signed hive Hive connector

Milestone

449

Development

Successfully merging this pull request may close these issues.

4 participants

@pajaks @findinpath @ebyhr @findepi