diff --git a/content/en/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes.md b/content/en/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes.md index 48f742a34..0d2325ec5 100644 --- a/content/en/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes.md +++ b/content/en/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes.md @@ -21,9 +21,9 @@ of [Vitess v2.0.0](https://github.com/youtube/vitess/releases). Some highlights * Java and Go clients use the new HTTP/2-based [gRPC](http://www.grpc.io/) framework. * Can now run on top of MySQL 5.6, in addition to MariaDB 10.0. * New administrative dashboard built on AngularJS. -* Built-in backup/restore](http://vitess.io/user-guide/backup-and-restore.html), designed to plug into blob stores like [Google Cloud Storage](https://cloud.google.com/storage/). -* GTID-based [reparenting](http://vitess.io/user-guide/reparenting.html) for reversible, routine failovers. -* Simpler [schema changes](http://vitess.io/user-guide/schema-management.html). +* [Built-in backup/restore](https://vitess.io/docs/user-guides/operating-vitess/backup-and-restore/) designed to plug into blob stores like [Google Cloud Storage](https://cloud.google.com/storage/). +* GTID-based [reparenting](https://vitess.io/docs/user-guides/configuration-advanced/reparenting/) for reversible, routine failovers. +* Simpler [schema changes](https://vitess.io/docs/reference/features/schema-management/). We've also been hard at work adding lots more [documentation](http://vitess.io/user-guide/introduction.html). In particular, the rest of this post will explore one of our new walkthroughs that demonstrates transparent [resharding](http://vitess.io/user-guide/sharding.html#resharding) of a live database - that is, changing the number of shards without any code changes or noticeable downtime for the application. @@ -38,30 +38,22 @@ values, which we call [keyspace IDs](http://vitess.io/overview/concepts.html#key **Transparent Resharding** -If you want to follow along with the new [resharding walkthrough](http://vitess.io/user-guide/sharding-kubernetes.html), you'll need to first bring up the cluster as described in the [unsharded guide](http://vitess.io/getting-started/). Both guides use the same [sample app](https://github.com/youtube/vitess/tree/master/examples/kubernetes/guestbook), which is a Guestbook that supports multiple, numbered pages. - -![Guestbook page 72 screenshot](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B5.png) +If you want to follow along with the new [resharding walkthrough](http://vitess.io/user-guide/sharding-kubernetes.html), you'll need to first bring up the cluster as described in the [unsharded guide](http://vitess.io/getting-started/). In the [sample app code](https://github.com/youtube/vitess/blob/master/examples/kubernetes/guestbook/main.py), you'll see a `get\_keyspace\_id()` function that transforms a given page number to the set of all 8-byte sequences, establishing the mapping we need for consistent hashing. In the unsharded case, these values are stored but not used. When we introduce sharding, page numbers will be evenly distributed (on average) across all the shards we create, allowing the app to scale to support arbitrary amounts of pages. Before resharding, you'll see a single [custom shard](http://vitess.io/user-guide/sharding.html#custom-sharding) named "0" in the Vitess dashboard. This is what an unsharded [keyspace](http://vitess.io/overview/concepts.html#keyspace) looks like. -![Guestbook dashboard screenshot](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B1.png) - As you begin the [resharding walkthrough](http://vitess.io/user-guide/sharding-kubernetes.html),you'll bring up two new shards for the same keyspace. During resharding, the new shards will run alongside the old one, but they'll remain idle (Vitess will not route any app traffic to them) until you're ready to migrate. In the dashboard, you'll see all three shards, but only shard "0" is currently active. -![Guestbook test_keyspace image](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B2.png) - Next, you'll run a few Vitess commands to [copy the schema and data](http://vitess.io/user-guide/sharding-kubernetes.html#copy-data-from-original-shard) from the original shard. The key to live migration is that once the initial snapshot copy is done, Vitess will automatically begin replicating fresh updates on the original shard to the new shards. We call this [filtered replication](http://vitess.io/user-guide/sharding.html#filtered-replication), since it distributes DMLs only to the shards to which they apply. Vitess also includes tools that compare the original and copied data sets, row-by-row, to [verify data integrity](http://vitess.io/user-guide/sharding-kubernetes.html#check-copied-data-integrity). Once you've verified the copy, and filtered replication has caught up to real-time updates, you can run the [migrate command](http://vitess.io/user-guide/sharding-kubernetes.html#switch-over-to-the-new-shards) which tells Vitess to atomically shift app traffic from the old shards to the new ones. It does this by disabling writes on the old masters, waiting for the new masters to receive the last events over filtered replication, and then enabling writes on the new masters. Since the process is automated, this typically only causes about a second of write unavailability. -Now you can[tear down the old shard](http://vitess.io/user-guide/sharding-kubernetes.html#remove-the-original-shard), +Now you can [tear down the old shard](http://vitess.io/user-guide/sharding-kubernetes.html#remove-the-original-shard), and verify that only the new ones show up in the dashboard. -![Guestbook dashboard image verify new shards](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B4.png) - Note that we never had to tell the app that we were changing from one shard to two. The resharding process was completely transparent to the app, since Vitess automatically reroutes queries on-the-fly as the migration progresses. At YouTube, we've used Vitess to transparently reshard (both [horizontally and vertically](http://vitess.io/user-guide/sharding.html#supported-operations)) nearly all of our MySQL databases within the last year alone, and we have still more on the horizon as we continue to grow. See the full [walkthrough instructions](http://vitess.io/user-guide/sharding-kubernetes.html) if you want to try it out for yourself. @@ -73,14 +65,10 @@ The promise of sharding is that it allows you to scale write throughput linearly Below you can see preliminary results for scaling write throughput by adding more shards in Vitess running on [Google Container Engine](https://cloud.google.com/container-engine/). For this benchmark, we pointed YCSB at the [load balancer](http://kubernetes.io/v1.0/docs/user-guide/services.html#type-loadbalancer) for our Vitess cluster and told it to send a lot of INSERT statements. Vitess took care of routing statements to the various shards. -[![](../images/thumbnails/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B3.png)](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B3.png) - The max throughput (QPS) for a given number of shards is the point at which round-trip write latency became degraded, which we define as >15ms on average or >50ms for the worst 1% of queries (99th percentile). We also ran YCSB's "read mostly" workload (95% reads, 5% writes) to show how Vitess can scale read traffic by adding replicas. The max throughput here is the point at which round-trip read latency became degraded, which we define as >5ms on average or >20ms for the worst 1% of queries. -![Guestbook image](../images/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes-vitess%2B6.png) - There's still a lot of room to improve the benchmarks (for example, by tuning the performance of MySQL itself). However, these preliminary results show that the returns don't diminish as you scale. And since you're scaling horizontally, you're not limited by the size of a single machine. **Conclusion** diff --git a/content/en/docs/contributing/code-reviews.md b/content/en/docs/contributing/code-reviews.md index d0fb2ddd5..14953957b 100644 --- a/content/en/docs/contributing/code-reviews.md +++ b/content/en/docs/contributing/code-reviews.md @@ -19,7 +19,7 @@ In order to avoid disruption, the following concerns need to be kept in mind: Every GitHub pull request must go through a code review and get approved before it will be merged into the master branch. Every pull request should meet the following requirements: -* Use the [Pull Request Template](https://github.com/vitessio/vitess/blob/master/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md) +* Use the [Pull Request Template](https://github.com/vitessio/vitess/blob/master/.github/pull_request_template.md) * Adhere to the [Go coding guidelines](https://golang.org/doc/effective_go.html) and watch out for these [common errors](https://github.com/golang/go/wiki/CodeReviewComments). * Contain a description message that is as detailed as possible. Here is a great example https://github.com/vitessio/vitess/pull/6543. * Pass all CI tests that run on PRs. diff --git a/content/en/docs/resources/presentations.md b/content/en/docs/resources/presentations.md index 705419471..1632acd2a 100644 --- a/content/en/docs/resources/presentations.md +++ b/content/en/docs/resources/presentations.md @@ -194,8 +194,9 @@ Vitess team member [Anthony Yeh](https://github.com/enisoc)'s talk at the [January 2016 CoreOS Meetup](http://www.meetup.com/coreos/events/228233948/) discussed challenges and techniques for running distributed databases within Kubernetes, followed by a deep dive into the design trade-offs -of the [Vitess on Kubernetes](https://github.com/vitessio/vitess/tree/master/examples/kubernetes) -deployment templates. +of the Vitess on Kubernetes deployment templates. + +Please note the Vitess on Kubernetes deployment templates were removed as of February 27, 2020. {{< pdf src="/ViewerJS/#../files/coreos-meetup-2016-01-27.pdf" >}} diff --git a/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md b/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md index 0535ae590..81c3e6dfb 100644 --- a/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md +++ b/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md @@ -1,7 +1,7 @@ --- title: Unmanaged Schema Changes weight: 3 -aliases: ['/docs/schema-management/unmanaged-schema-changes/', '/docs/user-guides/unmanaged-schema-changes/'] +aliases: ['/docs/user-guides/operating-vitess/making-schema-changes', '/docs/schema-management/unmanaged-schema-changes/', '/docs/user-guides/unmanaged-schema-changes/'] --- Vitess offers multiple approaches to running unmanaged schema changes. Below, we review each of these approaches.