Incubating Program: TiDB built-in SQL Diagnostics

[lonng]

Incubating Program

TiDB built-in SQL Diagnostics

Describe the feature or project you want to incubate:

Summary

Currently, TiDB diagnostic information acquisition relies mainly on external tools (perf/iosnoop/iotop/iostat/vmstat/sar/...), monitoring systems (Prometheus/Grafana), log files, HTTP APIs, and system tables provided by TiDB. The decentralized toolchains and cumbersome acquisition methods lead to high barriers to the use of TiDB clusters, difficulty in operation and maintenance, failure to detect problems in advance, and failure to timely investigate, diagnose, and recover clusters.
This proposal proposes a new method of acquiring diagnostic information in TiDB and exposing diagnostic information by the system tables so that users can query using SQL.

Motivation

This proposal mainly solves the following problems in TiDB's process of obtaining diagnostic information:

• The toolchains are scattered, it needs to switch back and forth between different tools, and some Linux distributions do not have built-in corresponding tools or built-in tools don't have versions as expected.
• The information acquisition methods are inconsistent, such as SQL, HTTP, export monitoring, login to each node to view logs, and so on.
• There are many TiDB cluster components, and the correlation monitoring information between different components is inefficient and cumbersome.
• TiDB does not have centralized log management components, and there is no efficient ways to filter, retrieve, analyze, and aggregate logs of the entire cluster.
• The system table only contains the current node information, and does not reflect the state of the entire cluster, such as: SLOW_QUERY, PROCESSLIST, STATEMENTS_SUMMARY.

The efficiency of the cluster-based information query, state acquisition, log retrieval, one-click inspection, and fault diagnosis will be improved after the multi-dimensional cluster-level system table and the cluster's diagnostic rule framework is provided. And provide basic data for the subsequent abnormal early warning function.

Estimated Time

30 days

Your RFC/Proposal?

pingcap/tidb#13481

[winkyao]

LGTM, I suggest that when the proposal is merged, this feature can be promoted to be an incubating feature.

[siddontang]

LGTM

[qiuyesuifeng]

LGTM

[Rustin170506]

/close

The issue has not been replied for a long time, so I will close the issue.

If the issue needs to be discussed, please reply to request reopen, and I will reopen the issue for you.

[ti-chi-bot]

@hi-rustin: Closing this issue.

In response to this:

/close

The issue has not been replied for a long time, so I will close the issue.

If the issue needs to be discussed, please reply to request reopen, and I will reopen the issue for you.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.