Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: disable merge queue in more tests #29235

Merged
merged 1 commit into from
Sep 4, 2018
Merged

Conversation

benesch
Copy link
Contributor

@benesch benesch commented Aug 29, 2018

Take a more aggressive approach to disabling the merge queue in tests.
This commit disables the merge queue in all tests that send a manual
AdminSplit request, unless those tests were explicitly verified to be
safe to run with the merge queue on.

Release note: None

Fix #29085.
Fix #29242.

@benesch benesch requested review from andreimatei, tbg, petermattis and a team August 29, 2018 03:42
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@benesch
Copy link
Contributor Author

benesch commented Aug 29, 2018

If this LGTY, please send this through Bors to reduce the flakiness!

@@ -471,6 +471,7 @@ func (db *DB) AdminMerge(ctx context.Context, key interface{}) error {
//
// The keys can be either byte slices or a strings.
func (db *DB) AdminSplit(ctx context.Context, spanKey, splitKey interface{}) error {
return errors.New("manual split")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left over?

Copy link
Contributor Author

@benesch benesch Aug 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hahaha, thanks. That's going to cause a test failure or fifty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: but please circle back to my comment about the skipped test.

bors r+

Reviewed 19 of 21 files at r1, 2 of 2 files at r2.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)


pkg/sql/distsql_physical_planner_test.go, line 99 at r2 (raw file):

func TestPlanningDuringSplitsAndMerges(t *testing.T) {
	defer leaktest.AfterTest(t)()
	t.Skip()

Add a comment. Or unskip this? Might be spurious.

@craig
Copy link
Contributor

craig bot commented Aug 29, 2018

Build failed

@tbg
Copy link
Member

tbg commented Aug 29, 2018

bors r+

TestLint failed with the url checker :(

@petermattis
Copy link
Collaborator

I didn't look at the contents of this PR (yet), but be aware that there is also Test{Cluster,Server}.SplitRange. Did you also look for tests using those calls?

@craig
Copy link
Contributor

craig bot commented Aug 29, 2018

Build failed (retrying...)

@tbg
Copy link
Member

tbg commented Aug 29, 2018

The current test failure looks like it's still caused by the merge queue:

https://teamcity.cockroachdb.com/viewLog.html?buildId=869108&buildTypeId=Cockroach_UnitTests

 TestImportPgDump/read_data_only

I'll merge the other PR. Let's smoke out the remaining flakes without messing up everyone else's day.

@craig
Copy link
Contributor

craig bot commented Aug 29, 2018

Build failed

@benesch
Copy link
Contributor Author

benesch commented Aug 29, 2018

I didn't look at the contents of this PR (yet), but be aware that there is also Test{Cluster,Server}.SplitRange. Did you also look for tests using those calls?

I did, yep!

TestImportPgDump is finding a real bug with merges, I think. I've plumbed in a DisableMergeQueue knob with a TODO for now.

craig bot pushed a commit that referenced this pull request Aug 29, 2018
29230: server: deflake TestRapidRestarts r=benesch a=petermattis

In go1.10 and earlier, it was not safe to call `http.ServeMux.ServeHTTP`
concurrently with `http.ServeMux.Handle`. (This is fixed in go1.11). In
the interim, provide our own safeServeMux wrapper that provides proper
locking.

Fixes #29227

Release note: None

29236: Revert "storage: enable the merge queue by default" r=tschottdorf a=benesch

This reverts commit 98ca1d0. The merge
queue will be reenabled once flaky tests are fixed.

To reviewers: I'd much prefer to merge #29235. But if that gets stuck in code review or the flakiness reaches a breaking point, feel free to merge this instead.

Co-authored-by: Peter Mattis <[email protected]>
Co-authored-by: Nikhil Benesch <[email protected]>
@benesch
Copy link
Contributor Author

benesch commented Aug 29, 2018

bors r=tschottdorf

Copy link
Contributor Author

@benesch benesch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)


pkg/sql/distsql_physical_planner_test.go, line 99 at r2 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Add a comment. Or unskip this? Might be spurious.

Oops, didn't mean to commit. It's removed in the rev that's getting bors'd.

Copy link
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you look at tests using kv.setupMultipleRanges()? E.g. TestTxnCoordSenderRetries should be in here but it's not.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)

Copy link
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, did you check if any flake issues can be closed? "No" is a good enough answer, but I'd like to know for when looking at the ones assigned to me.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)

@benesch
Copy link
Contributor Author

benesch commented Aug 29, 2018

Did you look at tests using kv.setupMultipleRanges()? E.g. TestTxnCoordSenderRetries should be in here but it's not.

Agh, I thought those were all covered by startNoSplitMergeServer. Looks like I missed these two

TestPropagateTxnOnError
TestTxnCoordSenderRetries

@benesch
Copy link
Contributor Author

benesch commented Aug 29, 2018

Also, did you check if any flake issues can be closed? "No" is a good enough answer, but I'd like to know for when looking at the ones assigned to me.

There was only the one I linked in the PR description when I opened this last night. I'll do another sweep sometime this afternoon.

Take a more aggressive approach to disabling the merge queue in tests.
This commit disables the merge queue in all tests that send a manual
AdminSplit request, unless those tests were explicitly verified to be
safe to run with the merge queue on.

Release note: None
@benesch
Copy link
Contributor Author

benesch commented Aug 30, 2018

This is contingent on #29324 going in first.

@benesch
Copy link
Contributor Author

benesch commented Sep 4, 2018

bors r=tschottdorf

1 similar comment
@benesch
Copy link
Contributor Author

benesch commented Sep 4, 2018

bors r=tschottdorf

craig bot pushed a commit that referenced this pull request Sep 4, 2018
29235: *: disable merge queue in more tests r=tschottdorf a=benesch

Take a more aggressive approach to disabling the merge queue in tests.
This commit disables the merge queue in all tests that send a manual
AdminSplit request, unless those tests were explicitly verified to be
safe to run with the merge queue on.

Release note: None

Fix #29085.
Fix #29242.

Co-authored-by: Nikhil Benesch <[email protected]>
@craig
Copy link
Contributor

craig bot commented Sep 4, 2018

Build succeeded

@craig craig bot merged commit cf83835 into cockroachdb:master Sep 4, 2018
@benesch benesch deleted the merge-flakes branch September 16, 2018 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants