Skip to content

Current ConnectionWorker retry strategy is too aggressive #2211

@mina-asham

Description

@mina-asham

Thanks for stopping by to let us know something could be better!

Environment details

  1. Specify the API at the beginning of the title. For example, "BigQuery: ..."): BigQuery
  2. OS type and version: ALL
  3. Java version: ALL
  4. version(s): Latest 2.41.0

Steps to reproduce

  1. Using Spark BigQuery connector
  2. Write a huge dataset to BigQuery
  3. Debug why it's super slow, the workers are waiting in com.google.cloud.bigquery.storage.v1.ConnectionWorker#calculateSleepTimeMilli

Looking at the code, the retry steps are too aggressive (1 milliseconds, 2 milliseconds, ..., 60 seconds), I suspect this is a bug and it was not intended to be this aggressive when retrying (the first 7 retries will happen in less than 256 milliseconds!)

Metadata

Metadata

Assignees

Labels

api: bigquerystorageIssues related to the googleapis/java-bigquerystorage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions