sql: track and display counters for successfully executed statements

[ajwerner]

Before this PR, statement counters were incremented before executing a SQL
statement. This is problematic because it means that the counters include
statements which fail during execution. This changes the logic to increment the
counters when statements complete successfully.

Release note (admin ui change): Only include successfully executed statements
in the statement counters.

[cockroach-teamcity]

This change is 

[ajwerner]

@piyush-singh I know we talked about this in the past, is there an issue I can link this to?

[piyush-singh]

#36126

[andreimatei]

I dunno bro, there's nothing in the metadata of those counters that suggests anything about "success".
I think we should have total counters and error counters, no?

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)

[ajwerner]

I dunno bro, there's nothing in the metadata of those counters that suggests anything about "success".
I think we should have total counters and error counters, no?

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)

Sure, it's just weird when you have a cluster where most of your requests are timing out and the topmost chart looks like your throughput is going up but in fact its going down. We don't have an easy way to subtract two time series from each other for display in the admin ui (a limitation of the tsdb I suppose).

How about I add two more metrics, failure and success for each statement type and then propose migrating the admin UI to the success metric?

[andreimatei]

Sure, it's just weird when you have a cluster where most of your requests are timing out and the topmost chart looks like your throughput is going up but in fact its going down.

This is only true for benchmark clients that spam the server with as many queries as they can, not for real apps where throughput presumably stays flat.

How about I add two more metrics, failure and success for each statement type and then propose migrating the admin UI to the success metric?

I think what I'd do is simply copy the errors metric to the overview metrics dashboard, just below SQL Queries.

[ajwerner]

I tried to do some digging on what others do. From a cursory glance I couldn't figure out the MySQL metric semantics. Here's what I dug out of some public grafana dashboard templates:

Cassandra: Uses count from their latency histogram which implies success
Postgres: Shows transaction commits and rollbacks and rows read and rows returned, nothing about number of statements.

I'm going to update this diff to track all 3 things and we can have a separate conversation with @piyush-singh about the right thing to display. I'll admit that this will collect redundant data but if having a few extra counters breaks the bank we've got bigger problems. Reasonable?

[ajwerner]

I'm realizing now that this also interacts with telemetry. Let me sync up with @piyush-singh before moving forward with this.

[ajwerner]

Still need to fix the testing, not yet ready for a look

[ajwerner]

This is RFAL. I attempted to do more programatic metric meta creation but ultimately reverted it in favor of just making another copy.

[andreimatei]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)

---

pkg/sql/conn_executor_exec.go, line 66 at r1 (raw file):

func (ex *connExecutor) execStmt(
	ctx context.Context, stmt Statement, res RestrictedCommandResult, pinfo *tree.PlaceholderInfo,
) (ev fsm.Event, payload fsm.EventPayload, err error) {

revert this

---

pkg/sql/conn_executor_exec.go, line 1305 at r1 (raw file):

}

// maybeIncrementExecutedStmtCounter checks if err and payload to determine if

s/checks if/checks

---

pkg/sql/conn_executor_exec.go, line 1308 at r1 (raw file):

// an error occurred and if not, increments the appropriate statement counter
// for stmt's type.
func (ex *connExecutor) maybeIncrementExecutedStmtCounter(

only one of the callers passes an err. And anything that deals with two possible errors is confusing. I'd remove the err from this function and deal with it in that one caller.
And then I won't nit about commenting `maybe..(.., nil /* err */)

[ajwerner]

TFTR! I moved the conditional logic into the caller

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)

---

pkg/sql/conn_executor_exec.go, line 66 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

revert this

Done.

---

pkg/sql/conn_executor_exec.go, line 1305 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

s/checks if/checks

Changed the logic.

---

pkg/sql/conn_executor_exec.go, line 1308 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

only one of the callers passes an err. And anything that deals with two possible errors is confusing. I'd remove the err from this function and deal with it in that one caller.
And then I won't nit about commenting `maybe..(.., nil /* err */)

I moved the conditional logic up to the callers.

[ajwerner]

I added some unit tests (certainly not as exhaustive as it could be) if you want take a look before I merge it

[ajwerner]

bors r+

[craig]

Build failed (retrying...)

• GitHub CI (Cockroach)

[craig]

Build succeeded

• GitHub CI (Cockroach)