Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add collector for database/sql#DBStats #866

Merged
merged 2 commits into from
May 20, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions prometheus/collectors/dbstats_collector.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
// Copyright 2021 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build go1.15
// +build go1.15

package collectors

import (
"database/sql"

"github.com/prometheus/client_golang/prometheus"
)

type dbStatsCollector struct {
db *sql.DB

maxOpenConnections *prometheus.Desc

openConnections *prometheus.Desc
inUseConnections *prometheus.Desc
idleConnections *prometheus.Desc

waitCount *prometheus.Desc
waitDuration *prometheus.Desc
maxIdleClosed *prometheus.Desc
maxIdleTimeClosed *prometheus.Desc
maxLifetimeClosed *prometheus.Desc
}

// DBStatsCollectorOpts defines the behavior of a db stats collector
// created with NewDBStatsCollector.
type DBStatsCollectorOpts struct {
// DriverName holds the name of driver.
// It will not used for empty strings.
DriverName string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just calling this Name or DBName? A user could create different *sql.DB instances with the same driver.

In different news: Do you anticipate that we need more fields in this options struct? An options struct is great for later non-breaking additions, but if those additions never happen, it's just overhead. I cannot think about anything needed here right now, so I would tend towards just having a dbName argument on the NewDBStatsCollector constructor.

}

// NewDBStatsCollector returns a collector that exports metrics about the given *sql.DB.
// See https://golang.org/pkg/database/sql/#DBStats for more information on stats.
func NewDBStatsCollector(db *sql.DB, opts DBStatsCollectorOpts) prometheus.Collector {
var fqName func(name string) string
if opts.DriverName == "" {
fqName = func(name string) string {
return "go_db_stats_" + name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about go_sql_ as prefix rather than go_db_stats_? The package it's coming from is called sql. And that kind of says everything. Obviously, it's about a database, and obviously, metrics ar "stats". go_sql_open_connections reads nicely, doesn't it?

}
} else {
fqName = func(name string) string {
return "go_" + opts.DriverName + "_db_stats_" + name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure the driver name should not be part of the metric name. It should be a label, perhaps called db_name.

That label has to be a const label when creating the descriptors, see comment below.

}
}
return &dbStatsCollector{
db: db,
maxOpenConnections: prometheus.NewDesc(
fqName("max_open_connections"),
"Maximum number of open connections to the database.",
nil, nil,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add the const label as suggested above, this line needs to be

    nil, prometheus.Labels{"db_name": opts.DriverName},

),
openConnections: prometheus.NewDesc(
fqName("open_connections"),
"The number of established connections both in use and idle.",
nil, nil,
),
inUseConnections: prometheus.NewDesc(
fqName("in_use_connections"),
"The number of connections currently in use.",
nil, nil,
),
idleConnections: prometheus.NewDesc(
fqName("idle_connections"),
"The number of idle connections.",
nil, nil,
),
waitCount: prometheus.NewDesc(
fqName("wait_count_total"),
"The total number of connections waited for.",
nil, nil,
),
waitDuration: prometheus.NewDesc(
fqName("wait_duration_seconds_total"),
"The total time blocked waiting for a new connection.",
nil, nil,
),
maxIdleClosed: prometheus.NewDesc(
fqName("max_idle_closed_total"),
"The total number of connections closed due to SetMaxIdleConns.",
nil, nil,
),
maxIdleTimeClosed: prometheus.NewDesc(
fqName("max_idle_time_closed_total"),
"The total number of connections closed due to SetConnMaxIdleTime.",
nil, nil,
),
maxLifetimeClosed: prometheus.NewDesc(
fqName("max_lifetime_closed_total"),
"The total number of connections closed due to SetConnMaxLifetime.",
nil, nil,
),
}
}

// Describe implements Collector.
func (c *dbStatsCollector) Describe(ch chan<- *prometheus.Desc) {
ch <- c.maxOpenConnections
ch <- c.openConnections
ch <- c.inUseConnections
ch <- c.idleConnections
ch <- c.waitCount
ch <- c.waitDuration
ch <- c.maxIdleClosed
ch <- c.maxIdleTimeClosed
ch <- c.maxLifetimeClosed
}

// Collect implements Collector.
func (c *dbStatsCollector) Collect(ch chan<- prometheus.Metric) {
stats := c.db.Stats()
ch <- prometheus.MustNewConstMetric(c.maxOpenConnections, prometheus.GaugeValue, float64(stats.MaxOpenConnections))
ch <- prometheus.MustNewConstMetric(c.openConnections, prometheus.GaugeValue, float64(stats.OpenConnections))
ch <- prometheus.MustNewConstMetric(c.inUseConnections, prometheus.GaugeValue, float64(stats.InUse))
ch <- prometheus.MustNewConstMetric(c.idleConnections, prometheus.GaugeValue, float64(stats.Idle))
ch <- prometheus.MustNewConstMetric(c.waitCount, prometheus.CounterValue, float64(stats.WaitCount))
ch <- prometheus.MustNewConstMetric(c.waitDuration, prometheus.CounterValue, stats.WaitDuration.Seconds())
ch <- prometheus.MustNewConstMetric(c.maxIdleClosed, prometheus.CounterValue, float64(stats.MaxIdleClosed))
ch <- prometheus.MustNewConstMetric(c.maxIdleTimeClosed, prometheus.CounterValue, float64(stats.MaxIdleTimeClosed))
ch <- prometheus.MustNewConstMetric(c.maxLifetimeClosed, prometheus.CounterValue, float64(stats.MaxLifetimeClosed))
}
129 changes: 129 additions & 0 deletions prometheus/collectors/dbstats_collector_go115.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
// Copyright 2021 The Prometheus Authors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this file dbstats_collector_pre_1.15.go.

Also, I think, you can avoid duplicating most of the code here.

For example, you could have the Desc for MaxIdleTimeClosed in any case, but do not Describe it and do not Collect the corresponding metric. Instead, have additional method calls there DescribeGo115Stats and CollectGo115Stats, and then just have those methods here (as empty), and the proper methods in a separate file with // +build go1.15 as a tag.

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build !go1.15
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this for? Isn't the directive in the next line the proper one?

// +build !go1.15

package collectors

import (
"database/sql"

"github.com/prometheus/client_golang/prometheus"
)

type dbStatsCollector struct {
db *sql.DB

maxOpenConnections *prometheus.Desc

openConnections *prometheus.Desc
inUseConnections *prometheus.Desc
idleConnections *prometheus.Desc

waitCount *prometheus.Desc
waitDuration *prometheus.Desc
maxIdleClosed *prometheus.Desc
maxLifetimeClosed *prometheus.Desc
}

// DBStatsCollectorOpts defines the behavior of a db stats collector
// created with NewDBStatsCollector.
type DBStatsCollectorOpts struct {
// DriverName holds the name of driver.
// It will not used for empty strings.
DriverName string
}

// NewDBStatsCollector returns a collector that exports metrics about the given *sql.DB.
// See https://golang.org/pkg/database/sql/#DBStats for more information on stats.
func NewDBStatsCollector(db *sql.DB, opts DBStatsCollectorOpts) prometheus.Collector {
var fqName func(name string) string
if opts.DriverName == "" {
fqName = func(name string) string {
return "go_db_stats_" + name
}
} else {
fqName = func(name string) string {
return "go_" + opts.DriverName + "_db_stats_" + name
}
}
return &dbStatsCollector{
db: db,
maxOpenConnections: prometheus.NewDesc(
fqName("max_open_connections"),
"Maximum number of open connections to the database.",
nil, nil,
),
openConnections: prometheus.NewDesc(
fqName("open_connections"),
"The number of established connections both in use and idle.",
nil, nil,
),
inUseConnections: prometheus.NewDesc(
fqName("in_use_connections"),
"The number of connections currently in use.",
nil, nil,
),
idleConnections: prometheus.NewDesc(
fqName("idle_connections"),
"The number of idle connections.",
nil, nil,
),
waitCount: prometheus.NewDesc(
fqName("wait_count_total"),
"The total number of connections waited for.",
nil, nil,
),
waitDuration: prometheus.NewDesc(
fqName("wait_duration_seconds_total"),
"The total time blocked waiting for a new connection.",
nil, nil,
),
maxIdleClosed: prometheus.NewDesc(
fqName("max_idle_closed_total"),
"The total number of connections closed due to SetMaxIdleConns.",
nil, nil,
),
maxLifetimeClosed: prometheus.NewDesc(
fqName("max_lifetime_closed_total"),
"The total number of connections closed due to SetConnMaxLifetime.",
nil, nil,
),
}
}

// Describe implements Collector.
func (c *dbStatsCollector) Describe(ch chan<- *prometheus.Desc) {
ch <- c.maxOpenConnections
ch <- c.openConnections
ch <- c.inUseConnections
ch <- c.idleConnections
ch <- c.waitCount
ch <- c.waitDuration
ch <- c.maxIdleClosed
ch <- c.maxLifetimeClosed
}

// Collect implements Collector.
func (c *dbStatsCollector) Collect(ch chan<- prometheus.Metric) {
stats := c.db.Stats()
ch <- prometheus.MustNewConstMetric(c.maxOpenConnections, prometheus.GaugeValue, float64(stats.MaxOpenConnections))
ch <- prometheus.MustNewConstMetric(c.openConnections, prometheus.GaugeValue, float64(stats.OpenConnections))
ch <- prometheus.MustNewConstMetric(c.inUseConnections, prometheus.GaugeValue, float64(stats.InUse))
ch <- prometheus.MustNewConstMetric(c.idleConnections, prometheus.GaugeValue, float64(stats.Idle))
ch <- prometheus.MustNewConstMetric(c.waitCount, prometheus.CounterValue, float64(stats.WaitCount))
ch <- prometheus.MustNewConstMetric(c.waitDuration, prometheus.CounterValue, stats.WaitDuration.Seconds())
ch <- prometheus.MustNewConstMetric(c.maxIdleClosed, prometheus.CounterValue, float64(stats.MaxIdleClosed))
ch <- prometheus.MustNewConstMetric(c.maxLifetimeClosed, prometheus.CounterValue, float64(stats.MaxLifetimeClosed))
}
71 changes: 71 additions & 0 deletions prometheus/collectors/dbstats_collector_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
// Copyright 2021 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package collectors

import (
"database/sql"
"runtime"
"testing"

"github.com/prometheus/client_golang/prometheus"
)

func TestDBStatsCollector(t *testing.T) {
reg := prometheus.NewRegistry()
db := new(sql.DB)
opts := DBStatsCollectorOpts{DriverName: "test"}
if err := reg.Register(NewDBStatsCollector(db, opts)); err != nil {
t.Fatal(err)
}

mfs, err := reg.Gather()
if err != nil {
t.Fatal(err)
}

names := []string{
"go_test_db_stats_max_open_connections",
"go_test_db_stats_open_connections",
"go_test_db_stats_in_use_connections",
"go_test_db_stats_idle_connections",
"go_test_db_stats_wait_count_total",
"go_test_db_stats_wait_duration_seconds_total",
"go_test_db_stats_max_idle_closed_total",
"go_test_db_stats_max_lifetime_closed_total",
}
if runtime.Version() >= "go1.15" {
names = append(names, "go_test_db_stats_max_idle_time_closed_total")
}
type result struct {
found bool
}
results := make(map[string]result)
for _, name := range names {
results[name] = result{found: false}
}
for _, mf := range mfs {
for _, name := range names {
if name == mf.GetName() {
results[name] = result{found: true}
break
}
}
}

for name, result := range results {
if !result.found {
t.Errorf("%s not found", name)
}
}
}