This repository was archived by the owner on Dec 16, 2022. It is now read-only.
Slack vitess upstream sync 2018 09 06.r0#109
Merged
Conversation
… so that it doesn't get LRU evicted as part of the TabletPlanCache. Signed-off-by: Maggie Zhou <mzhou@slack-corp.com>
Instead of logging a warning for the stats types which are known to be incompatible with prometheus (e.g. String, Rates, etc), just silently skip them in the publish phase. At the same time remove the throttled logger and replace with a warning log in case new metric types are created which aren't covered as part of the backend. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Also minor comment changes as suggested in PR feedback. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Greg Bird <greg.bird@joltup.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
After passing off the information needed from a read buffer, we recycle it with recycleReadPacket(). We don't do the same with writes, which means we hang out to allocated memory (in the case of ephemeralWriteSingleBuffer and ephemeralWriteBigBuffer) until the next write on the connection needs to allocate a new buffer (i.e. not using the global buffer). In steady state, this means we will gradually accumulate large buffers without ever letting them be GCed. This is wasteful, so add an analagous recycleWritePacket(). Signed-off-by: Daniel Tahara <tahara@dropbox.com>
Fix a regression where the dbname flags were removed as part of the earlier cleanup and refactor. This is against policy since we want to keep the flags for now. Unlike the other user-specific db connection flags, this one does not actually do anything at runtime except set a "DeprecatedDBName" field in the DBConfig. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Extract the common functionality for formatting bind variables out from both vtgate and vttablet's query logging modules into a sqltypes.FormatBindVariables helper. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Change the internal interface for query logging so that it passes an io.Writer into the formatter function as opposed to producing a temporary string and calling io.WriteBytes. This should reduce some GC pressure and enable more efficient buffer management under heavy workloads. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Replace the output section with the more up-to-date format and add some explanatory text for how to interpret it. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Per vitessio#4127, SET TRANSACTION ... has different semantics in absence of {SESSION,GLOBAL} than does SET tx_isolation or SET tx_readonly. Modify the data structures to reflect that difference. Signed-off-by: Daniel Tahara <tahara@dropbox.com>
* Also fixes cleanup bug. It should revert to tabletType provided, not RDONLY. Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
the server doesn't preemptively close connections due to our client keepalive being more aggressive than the allowed server policy. Signed-off-by: Maggie Zhou <mzhou@slack-corp.com>
Signed-off-by: Maggie Zhou <mzhou@slack-corp.com>
Signed-off-by: Maggie Zhou <mzhou@slack-corp.com>
[MySQL][Conn] Recyle write buffers if allocated
…name-flag restore the deprecated user-specific "dbname" flags
…-efficiency improve query logging efficiency by passing io.Writer along
…ng-started-doc OSX getting started docs
update vtexplain docs
…lows-v2 Improvements to workflows v2
add a SIGHUP handler for reloading vttablet's table ACL file
[go/mysql] refactor query benchmark to use RunParallel
Fixes regression in parser
In some cases (versions?) mysqldump appears to emit default expressions with the parens in the CURRENT_TIMESTAMP call. This normalizes to no parens either way in the AST. Signed-off-by: David Taylor <tinystatemachine@gmail.com>
MySQL accepts index definitions without explicit names, e.g. 'UNIQUE KEY (a, b)'. Signed-off-by: David Taylor <tinystatemachine@gmail.com>
sqlparser: allow () following CURRENT_TIMESTAMP in column defs
sqlparser: make index names optional
Signed-off-by: Shlomi Noach <shlomi-noach@github.com>
Signed-off-by: Shlomi Noach <shlomi-noach@github.com>
For a high QPS application, the sqlparser generates a ton of garbage. ``` 622GB 26.33% 60.87% 880.50GB 37.28% vitess.io/vitess/go/vt/sqlparser.yyParse 212.57GB 9.00% 69.87% 258.49GB 10.94% vitess.io/vitess/go/vt/sqlparser.(*yyParserImpl).Parse ``` Most of this, per the heap profile is allocations of new `yyParserImpl` objects for every call to `yyParse`. This diff uses `sync.Pool` to mitigate the total allocation overhead. Modifies sql.y to fix a place where we took a reference to a stack variable rather than making an intermediate copy. Resolves vitessio#4174 Signed-off-by: Daniel Tahara <tahara@dropbox.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
We already use such strategy for read buffers, so it makes sense to do the same for writes. Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: Aaron Young <young@squareup.com>
This should be indicative for changing ephemeral packets allocation strategies Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
[go/mysql] use sync.Pool for write buffers
…t-v3 Flush binary logs while reparenting
…tion-bit-type Handle bit types properly so that UPDATE and DELETE work as expected during replication
[Sqlparser] Pool the yyParserImpls
Fixing PID file creation and deletion
…l_lookup Allow the option of using null values in vindex fields
[go/mysql] add benchmark for random query sizes
update docs to use new proxy url in k8s example
Signed-off-by: deepthi <deepthi@planetscale.com>
…tion-bit-type vitessio#4145 changes based on review
|
👍 |
1 similar comment
|
👍 |
rafael
approved these changes
Sep 11, 2018
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slack changes:
Split workflow changes by rafael
Other changes
Allow planned reparenting to work without master/avoid master flags
commit faa1a87, PR Move QueryStats tracking into the QueryEngine and out of the TabletPlan vitessio/vitess#4093 from tinyspeck/fix-query-counts-spikiness-move-cache
Move QueryStats tracking into the QueryEngine and out of the TabletPlan
commit 16c5dd5, PR improve error if topo server is not specified vitessio/vitess#4169 from tinyspeck/topo-server-better-error
improve error if topo server is not specified
commit 6a80713, PR add a callinfo implementation for the mysql server protocol vitessio/vitess#4153 from tinyspeck/mysql-server-callinfo
add a callinfo implementation for the mysql server protocol
commit 2c7bdd4, PR rework the grpc callinfo to include the peer remote address vitessio/vitess#4151 from tinyspeck/grpc-peer-callinfo
rework the grpc callinfo to include the peer remote address
commit d9f7390, PR add support for vtexplain to override the dbname target vitessio/vitess#4150 from tinyspeck/vtexplain-set-dbname
add support for vtexplain to override the dbname target
commit 99d7f9e, PR Fixes regression in parser vitessio/vitess#4172 from tinyspeck/fix-regresion-in-parser
Fixes regression in parser
commit 9d7888e, PR silently ignore certain unsupported metric types vitessio/vitess#3901 from tinyspeck/minor-prom-fixes
silently ignore certain unsupported metric types
commit 1ff8c32, PR improve query logging efficiency by passing io.Writer along vitessio/vitess#4122 from tinyspeck/improve-query-logging-efficiency
improve query logging efficiency by passing io.Writer along
commit cd50187, PR Add Vagrantfile vitessio/vitess#4112 from tinyspeck/add-vagrant-file
Add Vagrantfile
commit fc0eee0, PR Do not update replication data if sanity check fails vitessio/vitess#4102 from tinyspeck/fix-initialize-tablet
Do not update replication data if sanity check fails
commit ff2984a, PR Remove RefreshConfig() from backups vitessio/vitess#4101 from tinyspeck/revert-refresh-config
Remove RefreshConfig() from backups
commit cf6afb6, PR make the query log timestamps more parseable vitessio/vitess#4096 from tinyspeck/logstats-dates-should-be-parseable
make the query log timestamps more parseable
commit 1ff8c32, PR improve query logging efficiency by passing io.Writer along vitessio/vitess#4122 from tinyspeck/improve-query-logging-efficiency
improve query logging efficiency by passing io.Writer along
commit aa1d579, PR restore the deprecated user-specific "dbname" flags vitessio/vitess#4121 from tinyspeck/restore-deprecated-dbname-flag
restore the deprecated user-specific "dbname" flags
not-Slack changes:
commit 3c4723b, PR sqlparser: allow () following CURRENT_TIMESTAMP in column defs vitessio/vitess#4168 from dt/current-ts-parens
sqlparser: allow () following CURRENT_TIMESTAMP in column defs
commit 24d253e, PR [Sqlparser] Normalizer should skip TableName and ColName vitessio/vitess#4156 from danieltahara/walk-optimize
[Sqlparser] Normalizer should skip TableName and ColName
commit c17ab98, PR [Resource Pool] Shorten critical section for register/unregister vitessio/vitess#4140 from danieltahara/numbered-pool
[Resource Pool] Shorten critical section for register/unregister
commit 52d3d14, PR sqlparser: parse ON DELETE CASCADE and friends vitessio/vitess#4135 from dt/ondelete
sqlparser: parse ON DELETE CASCADE and friends
vreplication: change to use new _vt.vreplication
commit 83d309e (master), PR vtgate planbuilder: verify same vindex vitessio/vitess#4132 from sougou/vindex
vtgate planbuilder: verify same vindex
commit 84c1c80, PR [Sqlparser] SET TRANSACTION has special semantics vitessio/vitess#4128 from danieltahara/extract-set
[Sqlparser] SET TRANSACTION has special semantics
commit 650d17b, PR [MySQL][Conn] Recyle write buffers if allocated vitessio/vitess#4120 from danieltahara/recycle-write
[MySQL][Conn] Recyle write buffers if allocated
commit c32e436, PR support create table like statement vitessio/vitess#4111 from klkyy/create-like
[sqlparser] support create table like statement
commit d03dedc, PR sqlparser: add StrictDDL variant of ParseNext vitessio/vitess#4082 from dt/strict
sqlparser: add StrictDDL variant of ParseNext
commit 5501814, PR sqlparser: parse empty statements as nil vitessio/vitess#4084 from dt/empty
sqlparser: parse empty statements as nil
commit 5930793, PR sqlparser: capture FOREIGN KEY constraints in AST vitessio/vitess#4085 from dt/fk
sqlparser: capture FOREIGN KEY constraints in AST
commit 3a51649, PR sqlparser: parse LOCK and UNLOCK TABLES statements vitessio/vitess#4083 from dt/lock
Upgrade of packages (goyacc and protobuf)
⬆️ commit 6fe8fa8, PR vitessio#4106 from sougou/rebuild
⬆️ commit 22669e9, PR vitessio#4010 from dweitzman/protobuf_v1.1.0
Other changes
commit 791cb0c
PR #4145 changes based on review vitessio/vitess#4184 from planetscale/ds-filtered-replication-bit-type
commit b416c1f
PR update docs to use new proxy url in k8s example vitessio/vitess#4107 from gbird3/update-k8s-docs
docs update
commit 9dd7fac
PR [go/mysql] add benchmark for random query sizes vitessio/vitess#4180 from LK4D4/add_random_query
Adds a benchmark for random query sizes
commit 032dd80
PR Allow the option of using null values in vindex fields vitessio/vitess#4146 from eeSeeGee/young.20180820.use_null_lookup
Allow the option of using null values in vindex fields
commit fd435b5
PR Fixing PID file creation and deletion vitessio/vitess#4177 from github/pid-file-fix
Fixing PID file creation and deletion on all binaries to allow us to cleanly shutdown processes. (issue pid file recreated even if exists vitessio/vitess#4176)
commit 28a97fc
PR [Sqlparser] Pool the yyParserImpls vitessio/vitess#4175 from danieltahara/parser-pool fixes issue Make Parser Memory Usage More Efficient vitessio/vitess#4174
[Sqlparser] Pool the yyParserImpls to make the parser memory usage more efficient.
commit 6f8bf2b
PR Handle bit types properly so that UPDATE and DELETE work as expected during replication vitessio/vitess#4167 from planetscale/ds-filtered-replication-bit-type
Handle bit types properly so that UPDATE and DELETE work as expected during replication
commit e4d51cf
PR [go/mysql] use sync.Pool for write buffers vitessio/vitess#4162 from LK4D4/pool_write_buffers
[go/mysql] use sync.Pool for write buffers for mysql connections
commit ecd4cc7
PR sqlparser: make index names optional vitessio/vitess#4166 from dt/name-opt
sqlparser: make index names optional
commit 02e95a8
PR [go/mysql] refactor query benchmark to use RunParallel vitessio/vitess#4165 from LK4D4/refactor_benchmark
[go/mysql] refactor query benchmark to use RunParallel
commit 9565d22
PR add a SIGHUP handler for reloading vttablet's table ACL file vitessio/vitess#4164 from slanning/vttablet_reload_acl
add a SIGHUP handler for reloading vttablet's table ACL file
commit e55032e
PR Fix getGeneratedKeys when no insert during batch upsert vitessio/vitess#4136 from HubSpot/fix-batch-upsert-noinsert
[java] change: Fix getGeneratedKeys when no insert during batch upsert
commit 7b67852
PR Better error message when topology implementation is not found vitessio/vitess#4170 from systay/better-error-message
Better error message when topology implementation is not found
commit 659d934
PR Adds the TabletAlias to the vars page json. vitessio/vitess#4171 from mpawliszyn/mikepaw.add-tablet-alias-var
Adds the TabletAlias to the vars page json.
commit 620e05c
PR Delay VTGateConnection closure when rotating connections vitessio/vitess#4141 from HubSpot/delay-channel-closure
[jdbc] Delay VTGateConnection closure when rotating connections
commit c941a35
PR fix errors in protobuf python generated code by regenerating vitessio/vitess#4144 from planetscale/master
[python] fix errors in protobuf python generated code by regenerating
commit 6f79d72
PR Update compose example for new db users, as well as fixing up README vitessio/vitess#4143 from eeSeeGee/young.20180817.update_example
[examples/docs] Update compose example for new db users, as well as fixing up README
commit f58872e
PR Fixing flakiness of schema_swap_test.py. vitessio/vitess#4142 from alainjobart/export_internal_changes
Fixing flakiness of schema_swap_test.py.
commit c3159d9
PR vschema: allow pins in vschema vitessio/vitess#4134 from sougou/pin
vschema: allow pins in vschema
commit 57bd460
PR update vtexplain docs vitessio/vitess#4124 from vitessio/fix-vtexplain-docs
[docs] update vtexplain docs
commit cd0af94
PR OSX getting started docs vitessio/vitess#4123 from planetscale/ds-update-osx-getting-started-doc
[docs] OSX getting started docs
commit 672186c
PR Fix some flaky tests vitessio/vitess#4113 from sougou/flakes
Fix some flaky tests
commit d523741
PR Go 1.11 fix for certificate common name. vitessio/vitess#4114 from alainjobart/tls
Go 1.11 fix for certificate common name.
commit 153cb28
commit 11ecb52
PR Show vitess target vitessio/vitess#3962 from tirsen/show-vitess-target
Show vitess target
commit f3fa98d
PR sqlparser: add option to skip special comments vitessio/vitess#4092 from dt/skip-comment
sqlparser: add option to skip special comments
commit 2e94e3e
PR nomycnf: operate without mycnf more naturally vitessio/vitess#4094 from sougou/nomycnf
nomycnf: operate without mycnf more naturally
commit 11f4ef5
PR In java, proto3 strings are never null. vitessio/vitess#4091 from alainjobart/javaproto
In java, proto3 strings are never null.
commit fbb85d8
PR vtgate: fix panic on 'analyze table' vitessio/vitess#4090 from sougou/panic
vtgate: fix panic on 'analyze table'
commit 0dfff0e
PR nomycnf: simplify db parameters vitessio/vitess#4088 from sougou/nomycnf
nomycnf: simplify db parameters
sqlparser: parse LOCK and UNLOCK TABLES statements