Skip to content

faster status updates#2615

Merged
max-hoffman merged 9 commits intomainfrom
max/sync-status-updates
Aug 1, 2024
Merged

faster status updates#2615
max-hoffman merged 9 commits intomainfrom
max/sync-status-updates

Conversation

@max-hoffman
Copy link
Copy Markdown
Contributor

@max-hoffman max-hoffman commented Aug 1, 2024

System variables can be session, global, or both. sql.IncrementStatusVariable is a helper method that primarily helps the "both" category increment the global and session counters for certain variables. Threads_running is a global only variable that is incremented/decremented every begin/end query, and gets a lot of traffic. The old code used sql.IncrementStatusVariable to increment Threads_running, which was a particularly expensive way to increment a global var because (1) we'd make a new error for every call to the session updater, and (2) the extra map lookup is unnecessary. We don't do the extra map lookup now, and we weren't using the error return so I removed the return variable.

Note: this also refactors status variables to be explicitly initializated in the engine

bump/perf here: dolthub/dolt#8189

@max-hoffman max-hoffman requested a review from jycor August 1, 2024 17:56
Copy link
Copy Markdown
Contributor

@jycor jycor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of crazy how much perf this unlocks, good find

sql/core.go Outdated
@@ -713,7 +713,7 @@ type StatusVariableRegistry interface {
// SetGlobal sets the global value of the status variable with the given name, returns an error if the variable is SessionOnly scope
SetGlobal(name string, val interface{}) error
// IncrementGlobal increments the value of the status variable by the given integer value
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should document failure behavior, kind of important

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added note about session-only nooping

v, ok := g.varVals[name]
if !ok || v.Variable().GetScope() == sql.StatusVariableScope_Session {
return sql.ErrUnknownSystemVariable.New(name)
return
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appropriate to panic here? Basically always means a bug right?

Copy link
Copy Markdown
Contributor Author

@max-hoffman max-hoffman Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The globals list excludes sessionOnly*, so it's only ~297/678 of the total list. So we want to ideally not call this on session only, but noop if we do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The query type counters have both scopes unfortunately, so I had to leave those for now

@max-hoffman max-hoffman merged commit 5dac189 into main Aug 1, 2024
@max-hoffman max-hoffman deleted the max/sync-status-updates branch August 1, 2024 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants