Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/ccl: Unskip TestTenantStatusAPI/tenant_ranges/pagination #99054

Merged
merged 1 commit into from
Mar 21, 2023

Conversation

abarganier
Copy link
Contributor

Fixes: #92979

Previously, in #97386, we skipped test_tenant_ranges_pagination because it was marked as flaky.

The test makes a request for a single range and expects an offset of 1 back. It then uses this offset to request a second range, and expects an offset of 2. This means that the test requires at least 3 ranges to exist on the tenant.

The test was flaking on the assertion that the offset returned by the second request came back as 2. Instead, it was flaking when the offset came back as 0, which signifies that there are no more ranges to process.

We learned that the tenant create process has an asycnhronous splitting of ranges that occurs, which is what would lead to this sporadic scenario where not enough ranges existed (yet) for the test to succeed.

This patch updates the test with a testutils.SucceedsSoon clause that checks first that crdb_internal.ranges contains at least 3 ranges, prior to making the second request. This should provide sufficient time for the range split queue to be processed and eliminate the vast majority of these test flakes.

Release note: none

@abarganier abarganier requested review from a team March 20, 2023 20:09
@abarganier abarganier requested a review from a team as a code owner March 20, 2023 20:09
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Fixes: cockroachdb#92979

Previously, in cockroachdb#97386,
we skipped test_tenant_ranges_pagination because it was marked as flaky.

The test makes a request for a single range and expects an offset of `1`
back. It then uses this offset to request a second range, and expects
an offset of `2`. This means that the test requires at least 3 ranges
to exist on the tenant.

The test was flaking on the assertion that the offset returned by the
second request came back as `2`. Instead, it was flaking when the
offset came back as `0`, which signifies that there are no more
ranges to process.

We learned that the tenant create process has an asycnhronous splitting
of ranges that occurs, which is what would lead to this sporadic scenario
where not enough ranges existed (yet) for the test to succeed.

This patch updates the test with a `testutils.SucceedsSoon` clause that
checks first that `crdb_internal.ranges` contains at least 3 ranges,
prior to making the second request. This should provide sufficient time
for the range split queue to be processed and eliminate the vast majority
of these test flakes.

Release note: none
@abarganier
Copy link
Contributor Author

TFTR!

bors r=dhartunian

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build succeeded:

@craig craig bot merged commit 77029b1 into cockroachdb:master Mar 21, 2023
zachlite added a commit to zachlite/cockroach that referenced this pull request Jul 11, 2023
This test was mistakenly skipped in cockroachdb#105197, but
the flake was addressed in cockroachdb#99054.

The flake on the 22.2 release branch (tracked in cockroachdb#92382) was not novel,
but un-treated. I misinterpreted this, and thought the skip
on master would be for good measure, not realizing the flake was
already treated.

Epic: none
Release note: None
craig bot pushed a commit that referenced this pull request Jul 11, 2023
106631: statusccl: unskip testTenantRanges pagination r=zachlite a=zachlite

This test was mistakenly skipped in #105197, but
the flake was addressed in #99054.

The flake on the 22.2 release branch (tracked in #92382) was not novel, but untreated. I misinterpreted this, and thought the skip on master would be for good measure, not realizing the flake was already treated.

Epic: none
Release note: None

Co-authored-by: zachlite <[email protected]>
blathers-crl bot pushed a commit that referenced this pull request Jul 11, 2023
This test was mistakenly skipped in #105197, but
the flake was addressed in #99054.

The flake on the 22.2 release branch (tracked in #92382) was not novel,
but un-treated. I misinterpreted this, and thought the skip
on master would be for good measure, not realizing the flake was
already treated.

Epic: none
Release note: None
zachlite added a commit to zachlite/cockroach that referenced this pull request Jul 12, 2023
This is a 22.2 compatible work around for the flake fix introduced in cockroachdb#99054.

In 22.2, crdb_internal.ranges is not yet compatible with
secondary tenants, so instead we rely on TenantRanges
itself with the `limit` param omitted.

The method is the same: we count the number of ranges that belong
to the tenant and only proceed when we know there are enough ranges
for subsequent calls with pagination to succeed.

Resolves cockroachdb#92382
Epic: none
Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pkg/ccl/serverccl/statusccl/statusccl_test: TestTenantStatusAPI failed
3 participants