Skip to content
This repository was archived by the owner on Dec 16, 2022. It is now read-only.

Slack sync upstream 2018 02 14#73

Merged
rafael merged 102 commits intomasterfrom
slack-sync-upstream-2018-02-14
Feb 15, 2018
Merged

Slack sync upstream 2018 02 14#73
rafael merged 102 commits intomasterfrom
slack-sync-upstream-2018-02-14

Conversation

@demmer
Copy link
Copy Markdown
Collaborator

@demmer demmer commented Feb 15, 2018

Latest sync with upstream.

To prepare the branch I first reverted the cherry-picked PRs #68 #71 and #72 then merged upstream at a0eae48. The net result is that we are now fully in-sync with the latest upstream.

If we want to clean up the history we can also force-push to get back onto the same git path.

Rafael Chacon and others added 30 commits January 28, 2018 17:57
Co-authored-by: Michael Demmer <mdemmer@slack-corp.com>
This makes RefreshConfig match the behavior of ReinitConfig.
Since the initial InitConfig happens in mysqlctld, it's important that
vttablet requests mysqlctld to regenerate config remotely, or else it
may not use the same settings.
Necessary for conditionally enabling semi sync if greater than one
Also used when waiting to run InitShardMaster to make sure we don’t orphan any slaves
Also use calculated totalTabletCount
To always reflect our +pb rename on import.
sougou and others added 26 commits February 13, 2018 08:21
Change non-unique lookup vindexes to return a full keyrange if
no match is found. This mode can be used when adding a new vindex
to a column. With this flag turned on new entries will use the
lookup while old ones will result in a scatter.

Once the backfill is done, the flag can be turned off.
With the Update function now part of Lookup, such vindexes can
now find more efficient implementations than the previous
Delete and Create sequence.

Currently, the Update implementations just do the Delete and Create,
but we'll soon improve those.

This abstraction also allows some vindexes to reject Updates if it's
not supported.

While making changes, I found a couple of bugs that I fixed:
* updateChangedVindexes fromIds computation was incorrect. It would
  have fetched incorrect values if the table had multiple multi-column
  and owned lookup vindexes.
* The Values for changing columns in planbuilder was getting filled
  out in the SET order, and not in the column order of the vindex,
  which was causing the updates to set the wrong values.

The above are not practical use cases, but it's better to strive for
correctness.
Added autocommit support for lookup vindex. In this mode, as described
in vitessio#3642, inserts are executed as upserts, deletes are ignored, and
updates are disallowed.
scatterIfAbsent was an unsafe flag because it could return
incorrect results if a lookup vindex was partially populated.
The only safe behavior is to always return a full scatter until
backfill is fully completed.

The flag is accordingly renamed to write_only
The new behavior for autocommit has been updated to reflect comments
in vitessio#3642:
* We never delete
* All db operations use autocommit to prevent deadlocks
* Non-unique vindexes upsert
* Unique vindexes always insert
* Updates are allowed as long as the underlying operations succeed
Flagged by our internal tools.
Previously there was no way to tell if the resilient server had a
valid cached value from the debug UI, as any error would trump the
cached value.

To address this, display the current cached value (if any), the last
error (if any), as well as the cache freshness / TTL for all portions
of the debug status UI.
This enables simulation of an unresponsive topo service.
Although the existing code already checks the cache TTL before returning
the value, the cached entry could be displayed in the status UI past its
TTL which isn't what we want.
Previously the lastValueTime was only updated when the watch detected
a change to the value. This meant that when the watch failed, it would
not necessarily cache the value for the full TTL, since the time in
cache was computed from the last change, not from the time when the
watch stopped.

To address this, always update lastValueTime when the watch was running
but stops, regardless of whether or not it got a value or an error.
Also make a couple of other changes to the tracking of lastErrorTime
and the last value to make the status UI more reflective of the
underlying state of the cache.
Rework GetSrvKeyspaceNames to asynchronously spin up a separate goroutine
and drop the mutex before calling out to the topo service. This ensures
that a valid cached value will be returned immediately even if it is
time to refresh the value, and that the mutex isn't held while fetching.

Also rework GetSrvKeyspace so that the goroutine used for the watch is
spun up before the initial watch is established, following the same
pattern.

This way we can continue to return cached entries and the debug UI
doesn't get locked up even if the topo service is unresponsive.
Removing useless conversion, fixing govet warning in test.
…esilient-and-informative

make the resilient topo cache even more resilient and informative
…unsharded

For unsharded dml, fix inner subquery referencing outer symbol
v3: migration support for lookup vindexes
vttablet: improved ACL support
…8-02-07"

This reverts commit 713273a, reversing
changes made to edc17cd.
…8-02-07"

This reverts commit 0ce5f56, reversing
changes made to 713273a.
…ray-comment"

This reverts commit 80d72ac, reversing
changes made to 732edb1.
…-29"

This reverts commit d48e84a, reversing
changes made to 83866b3.
@rafael
Copy link
Copy Markdown

rafael commented Feb 15, 2018

LGTM 👍

@rafael rafael merged commit 406223d into master Feb 15, 2018
@rafael rafael deleted the slack-sync-upstream-2018-02-14 branch February 15, 2018 19:20
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants