New metricbeat module: pgbouncer

metricbeat/docs/modules/pgbouncer.asciidoc

@@ -0,0 +1,130 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+:modulename: pgbouncer
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/pgbouncer/_meta/docs.asciidoc
+
+
+[[metricbeat-module-pgbouncer]]
+== PGBouncer module
+
+include::{libbeat-dir}/shared/integration-link.asciidoc[]
+
+:modulename!:
+
+PgBouncer is a popular, lightweight connection pooler for PostgreSQL. This module allows you to monitor PgBouncer using Metricbeat, gathering metrics that help optimize performance and maintain stability.
+
+This module periodically fetches metrics from the PgBouncer poolers available at https://www.pgbouncer.org/.
+
+Default metricsets include:
+- `lists`: Provides a list of databases and users currently connected to PgBouncer.
+- `mem`: Monitors memory usage details within PgBouncer, helping detect memory leaks or spikes.
+- `pools`: Tracks the status of connection pools, including active and idle connections.
+- `stats`: Collects comprehensive statistics on database connections and queries processed.
+
+[float]
+=== Module-specific configuration notes
+
+When configuring the `hosts` option, you must use PostgreSQL URLs of the following format with PgBouncer database name included:
+
+[source,yaml]
+-----------------------------------
+[postgres://][user:pass@]host[:port]/pgbouncer[?options]
+-----------------------------------
+
+The URL can be as simple as:
+
+[source,yaml]
+----------------------------------------------------------------------
+- module: pgbouncer
+  hosts: ["postgres://localhost/pgbouncer"]
+----------------------------------------------------------------------
+
+Or more complex like:
+
+[source,yaml]
+----------------------------------------------------------------------
+- module: pgbouncer
+  hosts: ["postgres://localhost/pgbouncer:40001?sslmode=disable", "postgres://otherhost/pgbouncer:40001"]
+----------------------------------------------------------------------
+
+Usernames and passwords specified in the URL take precedence over those specified in the `username` and `password` configuration options.
+For security best practices, consider using environment variables to pass sensitive information securely.
+
+[source,yaml]
+----
+- module: postgresql
+  metricsets: ["pools"]
+  hosts: ["postgres://localhost/pgbouncer:6432"]
+  username: root
+  password: test
+----
+
+:edit_url:
+
+[float]
+=== Example configuration
+
+The PGBouncer module supports the standard configuration options that are described
+in <<configuration-metricbeat>>. Here is an example configuration:
+
+[source,yaml]
+----
+metricbeat.modules:
+- module: pgbouncer
+  enabled: true
+  metricsets:
+    # Monitors the current state of the connection pools including metrics like active connections, idle connections, waiting connections, and max allowed connections
+    # Essential for assessing load and capacity
+    - pools
+
+    # Provides comprehensive data on connections, pools, and queries including total queries, average query duration, and query rates
+    # Useful for performance tuning and identifying bottlenecks
+    - stats
+
+    # Details current activity of the PgBouncer service showing lists of databases and users
+    # Helps in auditing access and understanding user distribution
+    - lists
+
+    # Reports on the sizes of internal memory allocations to track memory usage and potential leaks, critical for maintaining system stability
+    - memory
+
+  period: 10s
+
+  # The host must be passed as PostgreSQL URL. Example:
+  # postgres://localhost/pgbouncer:6432?sslmode=disable
+  # The available parameters are documented here:
+  # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
+  # You have to specify the pgbouncer as the database name.
+  hosts: ["postgres://localhost/pgbouncer:6432"]
+
+  # Username to use when connecting to PostgreSQL. Empty by default.
+  #username: user
+
+  # Password to use when connecting to PostgreSQL. Empty by default.
+  #password: pass
+----
+
+[float]
+=== Metricsets
+
+The following metricsets are available:
+
+* <<metricbeat-metricset-pgbouncer-lists,lists>>
+
+* <<metricbeat-metricset-pgbouncer-mem,mem>>
+
+* <<metricbeat-metricset-pgbouncer-pools,pools>>
+
+* <<metricbeat-metricset-pgbouncer-stats,stats>>
+
+include::pgbouncer/lists.asciidoc[]
+
+include::pgbouncer/mem.asciidoc[]
+
+include::pgbouncer/pools.asciidoc[]
+
+include::pgbouncer/stats.asciidoc[]
+
+:edit_url!:

metricbeat/docs/modules/pgbouncer/lists.asciidoc

@@ -0,0 +1,27 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/pgbouncer/lists/_meta/docs.asciidoc
+
+
+[[metricbeat-metricset-pgbouncer-lists]]
+=== PGBouncer lists metricset
+
+include::../../../module/pgbouncer/lists/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+:edit_url:
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-pgbouncer,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/pgbouncer/lists/_meta/data.json[]
+----
+:edit_url!:

metricbeat/docs/modules/pgbouncer/mem.asciidoc

@@ -0,0 +1,27 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/pgbouncer/mem/_meta/docs.asciidoc
+
+
+[[metricbeat-metricset-pgbouncer-mem]]
+=== PGBouncer mem metricset
+
+include::../../../module/pgbouncer/mem/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+:edit_url:
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-pgbouncer,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/pgbouncer/mem/_meta/data.json[]
+----
+:edit_url!:

metricbeat/docs/modules/pgbouncer/pools.asciidoc

@@ -0,0 +1,27 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/pgbouncer/pools/_meta/docs.asciidoc
+
+
+[[metricbeat-metricset-pgbouncer-pools]]
+=== PGBouncer pools metricset
+
+include::../../../module/pgbouncer/pools/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+:edit_url:
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-pgbouncer,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/pgbouncer/pools/_meta/data.json[]
+----
+:edit_url!:

metricbeat/docs/modules/pgbouncer/stats.asciidoc

@@ -0,0 +1,27 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/pgbouncer/stats/_meta/docs.asciidoc
+
+
+[[metricbeat-metricset-pgbouncer-stats]]
+=== PGBouncer stats metricset
+
+include::../../../module/pgbouncer/stats/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+:edit_url:
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-pgbouncer,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/pgbouncer/stats/_meta/data.json[]
+----
+:edit_url!:

metricbeat/docs/modules_list.asciidoc

@@ -251,6 +251,11 @@ This file is generated! See scripts/mage/docs_collector.go
 |<<metricbeat-metricset-panw-routing,routing>> beta[]  
 |<<metricbeat-metricset-panw-system,system>> beta[]  
 |<<metricbeat-metricset-panw-vpn,vpn>> beta[]  
+|<<metricbeat-module-pgbouncer,PGBouncer>>     |image:./images/icon-no.png[No prebuilt dashboards]    |  
+.4+| .4+|  |<<metricbeat-metricset-pgbouncer-lists,lists>>   
+|<<metricbeat-metricset-pgbouncer-mem,mem>>   
+|<<metricbeat-metricset-pgbouncer-pools,pools>>   
+|<<metricbeat-metricset-pgbouncer-stats,stats>>   
 |<<metricbeat-module-php_fpm,PHP_FPM>>     |image:./images/icon-no.png[No prebuilt dashboards]    |  
 .2+| .2+|  |<<metricbeat-metricset-php_fpm-pool,pool>>   
 |<<metricbeat-metricset-php_fpm-process,process>>   
@@ -384,6 +389,7 @@ include::modules/nginx.asciidoc[]
 include::modules/openmetrics.asciidoc[]
 include::modules/oracle.asciidoc[]
 include::modules/panw.asciidoc[]
+include::modules/pgbouncer.asciidoc[]
 include::modules/php_fpm.asciidoc[]
 include::modules/postgresql.asciidoc[]
 include::modules/prometheus.asciidoc[]

metricbeat/metricbeat.reference.yml

@@ -861,6 +861,40 @@ metricbeat.modules:
     include: []
     exclude: []
 
+#------------------------------ PGBouncer Module ------------------------------
+- module: pgbouncer
+  enabled: true
+  metricsets:
+    # Monitors the current state of the connection pools including metrics like active connections, idle connections, waiting connections, and max allowed connections
+    # Essential for assessing load and capacity
+    - pools
+
+    # Provides comprehensive data on connections, pools, and queries including total queries, average query duration, and query rates
+    # Useful for performance tuning and identifying bottlenecks
+    - stats
+
+    # Details current activity of the PgBouncer service showing lists of databases and users
+    # Helps in auditing access and understanding user distribution
+    - lists
+
+    # Reports on the sizes of internal memory allocations to track memory usage and potential leaks, critical for maintaining system stability
+    - memory
+
+  period: 10s
+
+  # The host must be passed as PostgreSQL URL. Example:
+  # postgres://localhost/pgbouncer:6432?sslmode=disable
+  # The available parameters are documented here:
+  # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
+  # You have to specify the pgbouncer as the database name.
+  hosts: ["postgres://localhost/pgbouncer:6432"]
+
+  # Username to use when connecting to PostgreSQL. Empty by default.
+  #username: user
+
+  # Password to use when connecting to PostgreSQL. Empty by default.
+  #password: pass
+
 #------------------------------- PHP_FPM Module -------------------------------
 - module: php_fpm
   metricsets:

metricbeat/module/pgbouncer/_meta/Dockerfile

@@ -0,0 +1,5 @@
+FROM edoburu/pgbouncer
+COPY ./pgbouncer.ini /etc/pgbouncer/pgbouncer.ini
+COPY ./userlist.txt /etc/pgbouncer/userlist.txt
+ENTRYPOINT ["/entrypoint.sh"]
+CMD ["/usr/bin/pgbouncer", "/etc/pgbouncer/pgbouncer.ini"]

metricbeat/module/pgbouncer/_meta/config.reference.yml

@@ -0,0 +1,32 @@
+- module: pgbouncer
+  enabled: true
+  metricsets:
+    # Monitors the current state of the connection pools including metrics like active connections, idle connections, waiting connections, and max allowed connections
+    # Essential for assessing load and capacity
+    - pools
+
+    # Provides comprehensive data on connections, pools, and queries including total queries, average query duration, and query rates
+    # Useful for performance tuning and identifying bottlenecks
+    - stats
+
+    # Details current activity of the PgBouncer service showing lists of databases and users
+    # Helps in auditing access and understanding user distribution
+    - lists
+
+    # Reports on the sizes of internal memory allocations to track memory usage and potential leaks, critical for maintaining system stability
+    - memory
+
+  period: 10s
+
+  # The host must be passed as PostgreSQL URL. Example:
+  # postgres://localhost/pgbouncer:6432?sslmode=disable
+  # The available parameters are documented here:
+  # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters
+  # You have to specify the pgbouncer as the database name.
+  hosts: ["postgres://localhost/pgbouncer:6432"]
+
+  # Username to use when connecting to PostgreSQL. Empty by default.
+  #username: user
+
+  # Password to use when connecting to PostgreSQL. Empty by default.
+  #password: pass

metricbeat/module/pgbouncer/_meta/config.yml

@@ -0,0 +1,11 @@
+metricbeat.modules:
+- module: pgbouncer
+  # metricsets:
+  #   - stats
+  #   - lists
+  #   - pools
+  #   - mem
+  period: 10s
+  hosts: ["postgresql://localhost:6432/pgbouncer?sslmode=disable"]
+  # username: test
+  # password: password

metricbeat/module/pgbouncer/_meta/config_local.yml

@@ -0,0 +1,17 @@
+metricbeat.modules:
+- module: pgbouncer
+  metricsets:
+    - mem
+    - lists
+    - stats
+    - pools
+  period: 10s
+  hosts: ["postgresql://localhost:6432/pgbouncer?sslmode=disable"]
+  username: test
+  password: password
+output.elasticsearch:
+  hosts: ["http://localhost:9200"]
+  username: elastic
+  password: xxxx
+  indices:
+    - index: "test"