Skip to content

Close BigQueryWriteClient in BigQueryPageSink#19933

Merged
ebyhr merged 1 commit intomasterfrom
ebi/bigquery-writer-close
Nov 28, 2023
Merged

Close BigQueryWriteClient in BigQueryPageSink#19933
ebyhr merged 1 commit intomasterfrom
ebi/bigquery-writer-close

Conversation

@ebyhr
Copy link
Member

@ebyhr ebyhr commented Nov 28, 2023

Description

https://cloud.google.com/java/docs/reference/google-cloud-bigquerystorage/latest/com.google.cloud.bigquery.storage.v1.BigQueryWriteClient

Note: close() needs to be called on the BigQueryWriteClient object to clean up resources such as threads. In the example above, try-with-resources is used, which automatically calls close().

There were many error messages.

2023-11-28T08:29:47.2348239Z 2023-11-28T02:29:47.167-0600	ERROR	Query-20231128_082946_00774_4nskn-4231	io.grpc.internal.ManagedChannelOrphanWrapper	*~*~*~ Previous channel {0} was not shutdown properly!!! ~*~*~*
2023-11-28T08:29:47.2349674Z     Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
2023-11-28T08:29:47.2350470Z java.lang.RuntimeException: ManagedChannel allocation site
2023-11-28T08:29:47.2351651Z 	at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:102)
2023-11-28T08:29:47.2352822Z 	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:60)
2023-11-28T08:29:47.2353852Z 	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:51)
2023-11-28T08:29:47.2354877Z 	at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:655)
2023-11-28T08:29:47.2355831Z 	at io.grpc.ForwardingChannelBuilder2.build(ForwardingChannelBuilder2.java:261)
2023-11-28T08:29:47.2356997Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:425)
2023-11-28T08:29:47.2358097Z 	at com.google.api.gax.grpc.ChannelPool.<init>(ChannelPool.java:107)
2023-11-28T08:29:47.2358869Z 	at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
2023-11-28T08:29:47.2359927Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
2023-11-28T08:29:47.2361312Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
2023-11-28T08:29:47.2362527Z 	at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:236)
2023-11-28T08:29:47.2363554Z 	at com.google.cloud.bigquery.storage.v1.stub.GrpcBigQueryWriteStub.create(GrpcBigQueryWriteStub.java:132)
2023-11-28T08:29:47.2365172Z 	at com.google.cloud.bigquery.storage.v1.stub.BigQueryWriteStubSettings.createStub(BigQueryWriteStubSettings.java:148)
2023-11-28T08:29:47.2366400Z 	at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.<init>(BigQueryWriteClient.java:143)
2023-11-28T08:29:47.2367476Z 	at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.create(BigQueryWriteClient.java:125)
2023-11-28T08:29:47.2368584Z 	at io.trino.plugin.bigquery.BigQueryWriteClientFactory.create(BigQueryWriteClientFactory.java:46)
2023-11-28T08:29:47.2369725Z 	at io.trino.plugin.bigquery.BigQueryPageSinkProvider.createPageSink(BigQueryPageSinkProvider.java:45)
2023-11-28T08:29:47.2370739Z 	at io.trino.split.PageSinkManager.createPageSink(PageSinkManager.java:48)
2023-11-28T08:29:47.2371792Z 	at io.trino.operator.TableWriterOperator$TableWriterOperatorFactory.createPageSink(TableWriterOperator.java:123)
2023-11-28T08:29:47.2373021Z 	at io.trino.operator.TableWriterOperator$TableWriterOperatorFactory.createOperator(TableWriterOperator.java:117)
2023-11-28T08:29:47.2374045Z 	at io.trino.operator.DriverFactory.createDriver(DriverFactory.java:105)
2023-11-28T08:29:47.2375025Z 	at io.trino.execution.SqlTaskExecution$DriverSplitRunnerFactory.createDriver(SqlTaskExecution.java:665)
2023-11-28T08:29:47.2376105Z 	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:876)
2023-11-28T08:29:47.2377243Z 	at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
2023-11-28T08:29:47.2378508Z 	at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
2023-11-28T08:29:47.2379668Z 	at io.trino.$gen.Trino_testversion____20231128_081425_738.run(Unknown Source)
2023-11-28T08:29:47.2380746Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
2023-11-28T08:29:47.2381774Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
2023-11-28T08:29:47.2382586Z 	at java.base/java.lang.Thread.run(Thread.java:1583)

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@cla-bot cla-bot bot added the cla-signed label Nov 28, 2023
@ebyhr ebyhr requested review from hashhar and wendigo November 28, 2023 09:34
@github-actions github-actions bot added the bigquery BigQuery connector label Nov 28, 2023
@ebyhr ebyhr merged commit 5d40483 into master Nov 28, 2023
@ebyhr ebyhr deleted the ebi/bigquery-writer-close branch November 28, 2023 11:21
@github-actions github-actions bot added this to the 434 milestone Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bigquery BigQuery connector cla-signed

Development

Successfully merging this pull request may close these issues.

3 participants