Skip to content

Conversation

@ebyhr
Copy link
Member

@ebyhr ebyhr commented May 20, 2024

Description

Invalidate remoteDatasetIdCache when after creating or dropping a schema.
This PR improves SHOW SCHEMAS speed 60 seconds → ~1 second with 75 schemas.

Release notes

(x) Release notes are required, with the following suggested text:

# BigQuery
* Improve performance when listing schemas and `bigquery.case-insensitive-name-matching` is enabled. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label May 20, 2024
@ebyhr ebyhr requested review from hashhar, nineinchnick and wendigo May 20, 2024 12:26
@github-actions github-actions bot added the bigquery BigQuery connector label May 20, 2024
@ebyhr
Copy link
Member Author

ebyhr commented May 20, 2024

TestBigQueryMetadataCaching#testMetadataCaching failure looks related to this change. I will investigate tomorrow.

@ebyhr ebyhr force-pushed the ebi/bigquery-case branch from 5bc4868 to 66e004c Compare May 21, 2024 00:42
@ebyhr ebyhr force-pushed the ebi/bigquery-case branch from 66e004c to 31be759 Compare May 21, 2024 02:37
public void createSchema(DatasetInfo datasetInfo)
{
bigQuery.create(datasetInfo);
remoteDatasetIdCache.invalidate(datasetInfo.getDatasetId().getProject());
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wendigo @hashhar @nineinchnick I pushed another small commit. Could you please take another look?

@ebyhr ebyhr merged commit fb6807f into master May 21, 2024
@ebyhr ebyhr deleted the ebi/bigquery-case branch May 21, 2024 09:54
@github-actions github-actions bot added this to the 449 milestone May 21, 2024
Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bigquery BigQuery connector cla-signed

Development

Successfully merging this pull request may close these issues.

5 participants