From 1bd39e8c6a1bf8584f6df8561025bb9ca1e0e809 Mon Sep 17 00:00:00 2001 From: Jaime Yera Hidalgo <106755265+jaimeyh@users.noreply.github.com> Date: Thu, 22 Feb 2024 14:58:02 +0000 Subject: [PATCH] SECCOMP-27660: Fix vulnerabilities with upstream repo changes (#95) * Dashboard linting improvements for mixin Signed-off-by: Ryan J. Geyer * Convert pg_stat_database to new collector model Signed-off-by: Joe Adams * Capture usename and application_name for pg_stat_activity It is necessary to be able to exclude backups from long-running transaction alerts, as they are to be expected. With the current pg_stat_activity metric there is no ability to filter out specific users or application names. Resolves #668 Signed-off-by: cezmunsta * Fixed formatting Signed-off-by: cezmunsta * Update common Prometheus files Signed-off-by: prombot * probe: clean-up database connection after probe to prevent connection leak Signed-off-by: Kurtis Bass * Set gauge to 1 when collector is successful Signed-off-by: Julien Pivotto Signed-off-by: Khiem Doan * Add postgres 15 for CI test Signed-off-by: Khiem Doan * Add postgres 15 for CI test Signed-off-by: Khiem Doan * New unit value 64kB Signed-off-by: Oleksandr Mysyura * Update common Prometheus files Signed-off-by: prombot * Update exporter-toolkit Update to the latest exporter-toolkit * Enables multi-listener and systemd socket activation. * Bump Go to 1.19. * Remove `PG_EXPORTER_WEB_LISTEN_ADDRESS` env var because this is now a repeatable flag. Signed-off-by: SuperQ * go fmt Signed-off-by: SuperQ * adding codified functionality for logical replication metrics Signed-off-by: Zachary Caldarola * Bump github.com/prometheus/client_golang from 1.13.0 to 1.14.0 Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.13.0 to 1.14.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.13.0...v1.14.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] * Bump github.com/prometheus/common from 0.37.0 to 0.39.0 Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](https://github.com/prometheus/common/compare/v0.37.0...v0.39.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] * addressing comments Signed-off-by: Zachary Caldarola * more comments Signed-off-by: Zachary Caldarola * fmt Signed-off-by: Zachary Caldarola * typing Signed-off-by: Zachary Caldarola * fmt Signed-off-by: Zachary Caldarola * send stdout/stderr to syslog Signed-off-by: Mike * Update common Prometheus files Signed-off-by: prombot * Fix exclude-databases for collector package The pg_database collector was not respecting the --exclude-databases flag and causing problems where databases were not accessible. This now respects the list of databases to exclude. - Adjusts the Collector create func to take a config struct instead of a logger. This allows more changes like this in the future. I figured we would need to do this at some point but I wasn't sure if we could hold off. - Split the database size collection to a separate query when database is not excluded. - Comment some probe code that was not useful/accurate Signed-off-by: Joe Adams * Remove commented code Signed-off-by: Joe Adams * Remove more dead code Signed-off-by: Joe Adams * Update build * Update Go to 1.20. * Update golanci-lint. * Bump modules. * Update CI orb. * Fix up use of deprecated ioutil. Signed-off-by: SuperQ * Reduce cardinality of pg_stat_statements Make the example queries.yaml `pg_stat_statements` query safer. * Select the top 10% of queries by total query time. * Only expose the top 100 queries by total query time. * Keep only the most useful metrics. * Comment out the example by default. Fixes: https://github.com/prometheus-community/postgres_exporter/issues/549 Signed-off-by: SuperQ * Update changelog and version for v0.12.0 release Signed-off-by: Joe Adams * Update exporter-toolkit Updates the exporter-toolkit to the latest version * Adds new landing page feature. * Allow metrics path to be on `/`. Signed-off-by: SuperQ * Update common Prometheus files Signed-off-by: prombot * Fix column type for pg_replication_slots Change the data type of `active` from int64 to bool. The documentation confirms that this is a boolean field. https://www.postgresql.org/docs/current/view-pg-replication-slots.html fixes #769 Signed-off-by: Joe Adams * Update versions listed in the README Update the supported versions based on what we actually test in CI. Signed-off-by: SuperQ * Update README cli flags These have not been kept up to date. Signed-off-by: Joe Adams * Adjust log level for collector startup Since we support both multi-target and typical direct scrapes, either of these can fail and it is no longer an error. Signed-off-by: Joe Adams * Fix pg_setting different help values Signed-off-by: GitHub * Supports alternate postgres:// prefix in URLs Adds support for the alternate postgres:// prefix in URLs. It's maybe not the cleanest approach, but works. Hoping I can either get some pointers on a more appropriate patch, or that we could use this in the interim to unblock this use-case. Signed-off-by: Jack Wink <57678801+mothershipper@users.noreply.github.com> * Bump github.com/lib/pq from 1.10.7 to 1.10.9 Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.7 to 1.10.9. - [Release notes](https://github.com/lib/pq/releases) - [Commits](https://github.com/lib/pq/compare/v1.10.7...v1.10.9) --- updated-dependencies: - dependency-name: github.com/lib/pq dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] * Refactor collector descriptors Use individual collector metric descriptor vars to help avoid miss-mapped or unused metrics. Signed-off-by: SuperQ * Bump github.com/prometheus/common from 0.42.0 to 0.44.0 Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.42.0 to 0.44.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](https://github.com/prometheus/common/compare/v0.42.0...v0.44.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] * Update linting * Move errcheck exclude list to config file. * Enable revive linter * Fix up revive linting issues. Signed-off-by: SuperQ * Bump github.com/prometheus/exporter-toolkit from 0.9.1 to 0.10.0 Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.9.1 to 0.10.0. - [Release notes](https://github.com/prometheus/exporter-toolkit/releases) - [Changelog](https://github.com/prometheus/exporter-toolkit/blob/master/CHANGELOG.md) - [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.9.1...v0.10.0) --- updated-dependencies: - dependency-name: github.com/prometheus/exporter-toolkit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] * Move queries from queries.yaml to collectors (#801) Signed-off-by: Ben Kochie * Fix pg_stat_database collector The signature for creating a collector changed and CI didn't retrigger. Move metrics out of map and into individual vars. Signed-off-by: Joe Adams * Fix up collector registration (#812) Use const definitions to make collector registration consistent. * Use collector subsystem name consistently. * Fix up replication metric name unit. Signed-off-by: SuperQ * Update release info for v0.12.1 Signed-off-by: Joe Adams * Deprecate extend queries feature (#811) Mark the extend queries feature as deprecated in favor of recommending the sql_exporter. Signed-off-by: SuperQ * Update common Prometheus files Signed-off-by: prombot * Deprecate additional database features Now that we have deprecated extended queries we can deprecate related database features. * Deprecate flags/functions around auto discover databases. * Deprecate flags/functions for additional constant labels. Signed-off-by: SuperQ * Release v0.13.0 BREAKING CHANGES: Please note, the following features are deprecated and may be removed in a future release: - `auto-discover-databases` - `extend.query-path` - `constantLabels` - `exclude-databases` - `include-databases` This exporter is meant to monitor PostgresSQL servers, not the user data/databases. If you need a generic SQL report exporter https://github.com/burningalchemist/sql_exporter is recommended. * [CHANGE] Adjust log level for collector startup #784 * [CHANGE] Move queries from queries.yaml to collectors #801 * [CHANGE] Deprecate extend queries feature #811 * [CHANGE] Deprecate additional database features #815 * [CHANGE] Convert pg_stat_database to new collector #685 * [ENHANCEMENT] Supports alternate postgres:// prefix in URLs #787 * [BUGFIX] Fix pg_setting different help values #771 * [BUGFIX] Fix column type for pg_replication_slots #777 * [BUGFIX] Fix pg_stat_database collector #809 Signed-off-by: SuperQ * Add the instance struct to handle connections The intent is to use the instance struct to hold the connection to the database as well as metadata about the instance. Currently this metadata only includes the version of postgres for the instance which can be used in the collectors to decide what query to run. In the future this could hold more metadata but for now it keeps the Collector interface arguments to a reasonable number. Signed-off-by: Joe Adams * chore: fix a few typos Signed-off-by: Alex Tymchuk * Bug fix: Make collector not fail on null values (#823) * Make all values nullable --------- Signed-off-by: Felix Yuan Co-authored-by: Ben Kochie * Release 0.13.1 (#824) * [BUGFIX] Make collectors not fail on null values #823 Signed-off-by: SuperQ * Fixed replication pgReplicationSlotQuery - now it's working correctly for replica and primary (#825) Signed-off-by: Vadim Voitenko Co-authored-by: Vadim Voitenko * Migrate pg_locks to collector package (#817) Migrate the `pg_locks_count` query from `main` to the `collector` package. Signed-off-by: SuperQ * Cleanup collectors (#826) Fix up `replication` and `process_idle` Update input params to match the rest of the collectors. Signed-off-by: SuperQ * Bug Fix: Fix lingering type issues (#828) * Fix postmaster type issue * Disable postmaster collector by default --------- Signed-off-by: Felix Yuan * Update common Prometheus files (#829) Signed-off-by: prombot * Fix replication collector Signed-off-by: Tom Hughes * Add some more escapes to the query sanitizer Signed-off-by: Tom Hughes * Add a collector to gather metrics on WAL size Signed-off-by: Tom Hughes * Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#853) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix untyped integer overflows on 32-bit archs (#857) go-sqlmock's Rows.AddRow() takes values which have a type alias of "any", and appear to default to untyped ints if not explicitly cast. When large values are passed which would overflow int32, tests fail. Signed-off-by: Daniel Swarbrick * Bump github.com/smartystreets/goconvey from 1.8.0 to 1.8.1 (#852) Bumps [github.com/smartystreets/goconvey](https://github.com/smartystreets/goconvey) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/smartystreets/goconvey/releases) - [Commits](https://github.com/smartystreets/goconvey/compare/v1.8.0...v1.8.1) --- updated-dependencies: - dependency-name: github.com/smartystreets/goconvey dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Unpack postgres arrays for process idle times correctly (#855) Signed-off-by: Ben Kochie * Include all idle processes in the process idle metrics Signed-off-by: Tom Hughes * Improve linting (#861) * Disable unused-parameter check due to false positives on Collect() calls. * Enable misspell. * Simplify error returns. Signed-off-by: SuperQ * Update common Prometheus files (#860) Signed-off-by: prombot Co-authored-by: Ben Kochie * Update common Prometheus files Signed-off-by: prombot * Gitlab collector: Database wraparound collector and test (#834) * Database wraparound collector and test --------- Signed-off-by: Felix Yuan Co-authored-by: Joe Adams * Add a logger to stat_database collector to get better handle on error (also clean up some metric validity checks) Signed-off-by: Felix Yuan * Update changelog for release 0.13.2 (#872) Signed-off-by: Joe Adams * Gitlab Collector: Autovacuum collector and test (#840) * Autovacuum collector and test Signed-off-by: Felix Yuan * Update collector/pg_stat_activity_autovacuum.go Co-authored-by: Joe Adams Signed-off-by: Felix Yuan * Update collector/pg_stat_activity_autovacuum.go Co-authored-by: Joe Adams Signed-off-by: Felix Yuan * Use timestamp seconds Signed-off-by: Felix Yuan * query formating Signed-off-by: Felix Yuan * SQL format Signed-off-by: Felix Yuan * Loosen autovacuum query Signed-off-by: Felix Yuan --------- Signed-off-by: Felix Yuan Co-authored-by: Joe Adams * Gitlab Collector: Wal Receiver Collector and Test (#844) * Wal Receiver Collector and Test Signed-off-by: Felix Yuan * Add more escapes Signed-off-by: Felix Yuan * Corrections to wal_receiver Signed-off-by: Felix Yuan * Continue on null labels Signed-off-by: Felix Yuan * Skip nulls and log a message Signed-off-by: Felix Yuan * Redundant breaks Signed-off-by: Felix Yuan * Fix up walreceiver Signed-off-by: Felix Yuan * Remove extra label Signed-off-by: Felix Yuan * Update collector/pg_stat_walreceiver.go Co-authored-by: Ben Kochie Signed-off-by: Felix Yuan * Clean up the extra assignments Signed-off-by: Felix Yuan * Update collector/pg_stat_walreceiver.go Co-authored-by: Joe Adams Signed-off-by: Felix Yuan --------- Signed-off-by: Felix Yuan Co-authored-by: Ben Kochie Co-authored-by: Joe Adams * Gitlab collector: Xlog location collector and test (#849) * Xlog location collector and test Signed-off-by: Felix Yuan * Add more escapes Signed-off-by: Felix Yuan * Change to Gauge Signed-off-by: Felix Yuan --------- Signed-off-by: Felix Yuan * Handle new pg_stat_statements column names (#874) Update pg_stat_statements collector to handle the new column names in PostgreSQL 13. Fixes: https://github.com/prometheus-community/postgres_exporter/issues/502 Signed-off-by: SuperQ * Fixup new pg_stats_statements query (#876) Fix all renames of `total_time` to `total_exec_time`. Fixes: https://github.com/prometheus-community/postgres_exporter/issues/502 Signed-off-by: SuperQ * Add a multi-target example config (#890) Add an example Prometheus scrape config, similar to the blackbox_exporter's example config. Fixes: https://github.com/prometheus-community/postgres_exporter/issues/888 Signed-off-by: SuperQ * Delay database connection until scrape (#882) This no longer returns an error when creating a collector.instance when the database cannot be reached for the version query. This will resolve the entire postgresCollector not being registered for metrics collection when a database is not available. If the version query fails, the scrape will fail. Resolves #880 Signed-off-by: Joe Adams * Bugfix: Make statsreset nullable (#877) * Stats_reset as null seems to actually be legitimate for new databases, so don't fail for it --------- Signed-off-by: Felix Yuan Co-authored-by: Ben Kochie * Gitlab Collector: User Index io stats collector and test (#845) * User Index io stats collector and test --------- Signed-off-by: Felix Yuan * Update README to reflect changes made in #828 (#894) Signed-off-by: Mathis Raguin * Gitlab Collector: Long running transactions collector and test (#836) * Long running transactions collector and test --------- Signed-off-by: Felix Yuan Co-authored-by: Ben Kochie * Update common Prometheus files (#900) Signed-off-by: prombot * Fix a connection leak (#902) The leak was introduced in PR#882 Signed-off-by: Christian Albrecht Co-authored-by: Christian Albrecht * Fix cross-compilation command in README.md (#903) Signed-off-by: David Cook * fix pg_replication_lag_seconds (#895) Signed-off-by: Vladimir Luksha Co-authored-by: Vladimir Luksha * stat_user_tables: Add total size metric (#904) Signed-off-by: David Cook * Fix bugs mentioned in #908 (#910) * Fix bugs mentioned in #908 These collectors are disabled by default, so unless enabled, they are not tested regularly. Signed-off-by: Joe Adams --------- Signed-off-by: Joe Adams * Update common Prometheus files (#913) Signed-off-by: prombot * Add changelog for v0.14 (#906) * Add changelog for v0.14 - Add changelog entries since v0.13.2 - Update README with new options - Bump version file Signed-off-by: Joe Adams * Add changelog entry for #904 Signed-off-by: Joe Adams --------- Signed-off-by: Joe Adams * Adds 1kB and 2kB units (#915) Signed-off-by: Eric tyrrell * Add error log when probe collector creation fails (#918) Signed-off-by: Joe Adams * Fix test build failures on 32-bit arch again (#919) Another case of untyped integer overflows on 32-bit arch. Signed-off-by: Daniel Swarbrick * Add 32-bit testing to CI (#920) Run Go tests with 32-bit to validate value overflow. Signed-off-by: SuperQ * Bump github.com/prometheus/client_golang from 1.16.0 to 1.17.0 (#925) * Bump github.com/prometheus/client_golang from 1.16.0 to 1.17.0 Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.16.0 to 1.17.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.16.0...v1.17.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] * Update tests for latest client_golang. Signed-off-by: SuperQ --------- Signed-off-by: dependabot[bot] Signed-off-by: SuperQ Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: SuperQ * Update common Prometheus files (#926) Signed-off-by: prombot * Adjust collector to use separate connection per scrape (#931) Fixes #921 Signed-off-by: Joe Adams * Bump golang.org/x/net from 0.10.0 to 0.17.0 (#936) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.17.0. - [Commits](https://github.com/golang/net/compare/v0.10.0...v0.17.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Release v0.15.0 (#944) * [ENHANCEMENT] Add 1kB and 2kB units #915 * [BUGFIX] Add error log when probe collector creation fails #918 * [BUGFIX] Fix test build failures on 32-bit arch #919 * [BUGFIX] Adjust collector to use separate connection per scrape #936 Signed-off-by: SuperQ * Update common Prometheus files (#951) Signed-off-by: prombot * Update common Prometheus files (#963) Signed-off-by: prombot * pg_replication_slot: add slot type label (#960) Signed-off-by: Alex Simenduev * Bump github.com/prometheus/common from 0.44.0 to 0.45.0 (#948) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.44.0 to 0.45.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](https://github.com/prometheus/common/compare/v0.44.0...v0.45.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/client_model (#949) Bumps [github.com/prometheus/client_model](https://github.com/prometheus/client_model) from 0.4.1-0.20230718164431-9a2bf3000d16 to 0.5.0. - [Release notes](https://github.com/prometheus/client_model/releases) - [Commits](https://github.com/prometheus/client_model/commits/v0.5.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_model dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * pg_stat_database: added support for `active_time` counter (#961) * feat(pg_stat_database): active time metric --------- Signed-off-by: Jiri Sveceny * Bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#988) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0. - [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 (#993) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.17.0 to 1.18.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.17.0...v1.18.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * use Info level for excluded databases log message (#1003) This is the only log message which didn't specify a level in the postgres_exporter. I am unsure if this log message should be info or debug, but leaning towards the more important since previously it would just always log. The way I validated this was the only non-leveled logger was via grep. Both of these only returned this callsite previously: git grep 'logger\.Log' git grep '\.Log(' | grep -v level Signed-off-by: Keegan Carruthers-Smith --------- Signed-off-by: Ryan J. Geyer Signed-off-by: Joe Adams Signed-off-by: cezmunsta Signed-off-by: prombot Signed-off-by: Kurtis Bass Signed-off-by: Julien Pivotto Signed-off-by: Khiem Doan Signed-off-by: Oleksandr Mysyura Signed-off-by: SuperQ Signed-off-by: Zachary Caldarola Signed-off-by: dependabot[bot] Signed-off-by: Zachary Caldarola Signed-off-by: Mike Signed-off-by: GitHub Signed-off-by: Jack Wink <57678801+mothershipper@users.noreply.github.com> Signed-off-by: Ben Kochie Signed-off-by: Alex Tymchuk Signed-off-by: Felix Yuan Signed-off-by: Vadim Voitenko Signed-off-by: Tom Hughes Signed-off-by: Daniel Swarbrick Signed-off-by: Mathis Raguin Signed-off-by: Christian Albrecht Signed-off-by: David Cook Signed-off-by: Vladimir Luksha Signed-off-by: Eric tyrrell Signed-off-by: Alex Simenduev Signed-off-by: Jiri Sveceny Signed-off-by: Keegan Carruthers-Smith Co-authored-by: Ryan J. Geyer Co-authored-by: Joe Adams Co-authored-by: cezmunsta Co-authored-by: prombot Co-authored-by: Kurtis Bass Co-authored-by: Julien Pivotto Co-authored-by: Khiem Doan Co-authored-by: Oleksandr Mysyura Co-authored-by: Ben Kochie Co-authored-by: Zachary Caldarola Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Zachary Caldarola Co-authored-by: Mike Co-authored-by: Khaled Khalifa <33331600+khkhalifa@users.noreply.github.com> Co-authored-by: Jack Wink <57678801+mothershipper@users.noreply.github.com> Co-authored-by: Felix Yuan Co-authored-by: Alex Tymchuk Co-authored-by: Vadim Voitenko <74241416+wwoytenko@users.noreply.github.com> Co-authored-by: Vadim Voitenko Co-authored-by: Tom Hughes Co-authored-by: Daniel Swarbrick Co-authored-by: Mathis Raguin Co-authored-by: Christian Albrecht Co-authored-by: Christian Albrecht Co-authored-by: David Cook Co-authored-by: Vladimir Luksha Co-authored-by: Vladimir Luksha Co-authored-by: David Cook Co-authored-by: Eric Tyrrell <58529434+Eric-Tyrrell22@users.noreply.github.com> Co-authored-by: Alex Simenduev Co-authored-by: Jiri Sveceny Co-authored-by: Keegan Carruthers-Smith --- .circleci/config.yml | 12 +- .github/workflows/golangci-lint.yml | 12 +- .golangci.yml | 15 +- .promu.yml | 3 +- .yamllint | 7 +- CHANGELOG.md | 68 ++- Makefile.common | 22 +- README.md | 122 +++- VERSION | 2 +- cmd/postgres_exporter/datasource.go | 9 +- cmd/postgres_exporter/main.go | 89 +-- cmd/postgres_exporter/pg_setting.go | 10 +- cmd/postgres_exporter/pg_setting_test.go | 22 +- cmd/postgres_exporter/postgres_exporter.go | 70 +-- .../postgres_exporter_test.go | 3 +- cmd/postgres_exporter/probe.go | 29 +- cmd/postgres_exporter/queries.go | 35 +- collector/collector.go | 52 +- collector/collector_test.go | 62 ++ collector/instance.go | 106 ++++ collector/pg_database.go | 100 +++- collector/pg_database_test.go | 101 ++++ collector/pg_database_wraparound.go | 115 ++++ collector/pg_database_wraparound_test.go | 64 +++ collector/pg_locks.go | 129 +++++ collector/pg_locks_test.go | 60 ++ collector/pg_long_running_transactions.go | 93 +++ .../pg_long_running_transactions_test.go | 63 ++ collector/pg_postmaster.go | 69 +++ collector/pg_postmaster_test.go | 95 ++++ collector/pg_process_idle.go | 132 +++++ collector/pg_replication.go | 88 +++ collector/pg_replication_slot.go | 136 +++++ collector/pg_replication_slot_test.go | 186 ++++++ collector/pg_replication_test.go | 63 ++ collector/pg_stat_activity_autovacuum.go | 84 +++ collector/pg_stat_activity_autovacuum_test.go | 62 ++ collector/pg_stat_bgwriter.go | 191 ++++--- collector/pg_stat_bgwriter_test.go | 151 +++++ collector/pg_stat_database.go | 500 ++++++++++++++++ collector/pg_stat_database_test.go | 523 +++++++++++++++++ collector/pg_stat_statements.go | 211 +++++++ collector/pg_stat_statements_test.go | 153 +++++ collector/pg_stat_user_tables.go | 446 +++++++++++++++ collector/pg_stat_user_tables_test.go | 243 ++++++++ collector/pg_stat_walreceiver.go | 271 +++++++++ collector/pg_stat_walreceiver_test.go | 186 ++++++ collector/pg_statio_user_indexes.go | 118 ++++ collector/pg_statio_user_indexes_test.go | 109 ++++ collector/pg_statio_user_tables.go | 222 ++++++++ collector/pg_statio_user_tables_test.go | 157 +++++ collector/pg_wal.go | 84 +++ collector/pg_wal_test.go | 63 ++ collector/pg_xlog_location.go | 91 +++ collector/pg_xlog_location_test.go | 61 ++ collector/probe.go | 32 +- config/config.go | 6 +- config/config_test.go | 4 +- config/dsn.go | 2 +- config/dsn_test.go | 13 + go.mod | 53 +- go.sum | 536 +++--------------- postgres_exporter.rc | 3 +- .../dashboards/postgres-overview.json | 140 ++--- queries.yaml | 265 +-------- scripts/errcheck_excludes.txt | 2 - 66 files changed, 6041 insertions(+), 1185 deletions(-) create mode 100644 collector/collector_test.go create mode 100644 collector/instance.go create mode 100644 collector/pg_database_test.go create mode 100644 collector/pg_database_wraparound.go create mode 100644 collector/pg_database_wraparound_test.go create mode 100644 collector/pg_locks.go create mode 100644 collector/pg_locks_test.go create mode 100644 collector/pg_long_running_transactions.go create mode 100644 collector/pg_long_running_transactions_test.go create mode 100644 collector/pg_postmaster.go create mode 100644 collector/pg_postmaster_test.go create mode 100644 collector/pg_process_idle.go create mode 100644 collector/pg_replication.go create mode 100644 collector/pg_replication_slot.go create mode 100644 collector/pg_replication_slot_test.go create mode 100644 collector/pg_replication_test.go create mode 100644 collector/pg_stat_activity_autovacuum.go create mode 100644 collector/pg_stat_activity_autovacuum_test.go create mode 100644 collector/pg_stat_bgwriter_test.go create mode 100644 collector/pg_stat_database.go create mode 100644 collector/pg_stat_database_test.go create mode 100644 collector/pg_stat_statements.go create mode 100644 collector/pg_stat_statements_test.go create mode 100644 collector/pg_stat_user_tables.go create mode 100644 collector/pg_stat_user_tables_test.go create mode 100644 collector/pg_stat_walreceiver.go create mode 100644 collector/pg_stat_walreceiver_test.go create mode 100644 collector/pg_statio_user_indexes.go create mode 100644 collector/pg_statio_user_indexes_test.go create mode 100644 collector/pg_statio_user_tables.go create mode 100644 collector/pg_statio_user_tables_test.go create mode 100644 collector/pg_wal.go create mode 100644 collector/pg_wal_test.go create mode 100644 collector/pg_xlog_location.go create mode 100644 collector/pg_xlog_location_test.go delete mode 100644 scripts/errcheck_excludes.txt diff --git a/.circleci/config.yml b/.circleci/config.yml index ae168f04e..98099e295 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -2,13 +2,13 @@ version: 2.1 orbs: - prometheus: prometheus/prometheus@0.16.0 + prometheus: prometheus/prometheus@0.17.1 executors: # This must match .promu.yml. golang: docker: - - image: cimg/go:1.18 + - image: cimg/go:1.21 jobs: test: @@ -16,13 +16,14 @@ jobs: steps: - prometheus/setup_environment + - run: GOHOSTARCH=386 GOARCH=386 make test - run: make - prometheus/store_artifact: file: postgres_exporter integration: docker: - - image: cimg/go:1.18 + - image: cimg/go:1.20 - image: << parameters.postgres_image >> environment: POSTGRES_DB: circle_test @@ -56,11 +57,12 @@ workflows: matrix: parameters: postgres_image: - - circleci/postgres:10 - circleci/postgres:11 - circleci/postgres:12 - circleci/postgres:13 - - cimg/postgres:14.1 + - cimg/postgres:14.9 + - cimg/postgres:15.4 + - cimg/postgres:16.0 - prometheus/build: name: build parallelism: 3 diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index feada6f53..ffa6b3090 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -1,3 +1,5 @@ +--- +# This action is synced from https://github.com/prometheus/prometheus name: golangci-lint on: push: @@ -16,15 +18,15 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout repository - uses: actions/checkout@v3 + uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 - name: install Go - uses: actions/setup-go@v4 + uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # v3.5.0 with: - go-version: 1.18.x + go-version: 1.21.x - name: Install snmp_exporter/generator dependencies run: sudo apt-get update && sudo apt-get -y install libsnmp-dev if: github.repository == 'prometheus/snmp_exporter' - name: Lint - uses: golangci/golangci-lint-action@v3.4.0 + uses: golangci/golangci-lint-action@3a919529898de77ec3da873e3063ca4b10e7f5cc # v3.7.0 with: - version: v1.52.2 + version: v1.54.2 diff --git a/.golangci.yml b/.golangci.yml index 096572c13..96487c898 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,4 +1,9 @@ --- +linters: + enable: + - misspell + - revive + issues: exclude-rules: - path: _test.go @@ -7,4 +12,12 @@ issues: linters-settings: errcheck: - exclude: scripts/errcheck_excludes.txt + exclude-functions: + # Never check for logger errors. + - (github.com/go-kit/log.Logger).Log + revive: + rules: + # https://github.com/mgechev/revive/blob/master/RULES_DESCRIPTIONS.md#unused-parameter + - name: unused-parameter + severity: warning + disabled: true diff --git a/.promu.yml b/.promu.yml index d63d0444d..dbad0ba95 100644 --- a/.promu.yml +++ b/.promu.yml @@ -1,13 +1,12 @@ go: # This must match .circle/config.yml. - version: 1.18 + version: 1.21 repository: path: github.com/prometheus-community/postgres_exporter build: binaries: - name: postgres_exporter path: ./cmd/postgres_exporter - flags: -a -tags 'netgo static_build' ldflags: | -X github.com/prometheus/common/version.Version={{.Version}} -X github.com/prometheus/common/version.Revision={{.Revision}} diff --git a/.yamllint b/.yamllint index 3878a31d3..955a5a627 100644 --- a/.yamllint +++ b/.yamllint @@ -20,9 +20,4 @@ rules: config/testdata/section_key_dup.bad.yml line-length: disable truthy: - ignore: | - .github/workflows/codeql-analysis.yml - .github/workflows/funcbench.yml - .github/workflows/fuzzing.yml - .github/workflows/prombench.yml - .github/workflows/golangci-lint.yml + check-keys: false diff --git a/CHANGELOG.md b/CHANGELOG.md index 38d7e4d90..4d17f536b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,66 @@ -## master / unreleased +## 0.15.0 / 2023-10-27 -## 0.12.0-rc.0 / 2022-08-26 +* [ENHANCEMENT] Add 1kB and 2kB units #915 +* [BUGFIX] Add error log when probe collector creation fails #918 +* [BUGFIX] Fix test build failures on 32-bit arch #919 +* [BUGFIX] Adjust collector to use separate connection per scrape #936 + +## 0.14.0 / 2023-09-11 + +* [CHANGE] Add `state` label to pg_process_idle_seconds #862 +* [CHANGE] Change database connections to one per scrape #882 #902 +* [ENHANCEMENT] Add wal collector #858 +* [ENHANCEMENT] Add database_wraparound collector #834 +* [ENHANCEMENT] Add stat_activity_autovacuum collector #840 +* [ENHANCEMENT] Add stat_wal_receiver collector #844 +* [ENHANCEMENT] Add xlog_location collector #849 +* [ENHANCEMENT] Add statio_user_indexes collector #845 +* [ENHANCEMENT] Add long_running_transactions collector #836 +* [ENHANCEMENT] Add pg_stat_user_tables_size_bytes metric #904 +* [BUGFIX] Fix tests on 32-bit systems #857 +* [BUGFIX] Fix pg_stat_statements metrics on Postgres 13+ #874 #876 +* [BUGFIX] Fix pg_stat_database metrics for NULL stats_reset #877 +* [BUGFIX] Fix pg_replication_lag_seconds on Postgres 10+ when master is idle #895 + +## 0.13.2 / 2023-07-21 + +* [BUGFIX] Fix type issues on pg_postmaster metrics #828 +* [BUGFIX] Fix pg_replication collector instantiation #854 +* [BUGFIX] Fix pg_process_idle metrics #855 + +## 0.13.1 / 2023-06-27 + +* [BUGFIX] Make collectors not fail on null values #823 + +## 0.13.0 / 2023-06-21 + +BREAKING CHANGES: + +Please note, the following features are deprecated and may be removed in a future release: +- `auto-discover-databases` +- `extend.query-path` +- `constantLabels` +- `exclude-databases` +- `include-databases` + +This exporter is meant to monitor PostgresSQL servers, not the user data/databases. If +you need a generic SQL report exporter https://github.com/burningalchemist/sql_exporter +is recommended. + +* [CHANGE] Adjust log level for collector startup #784 +* [CHANGE] Move queries from queries.yaml to collectors #801 +* [CHANGE] Deprecate extend queries feature #811 +* [CHANGE] Deprecate additional database features #815 +* [CHANGE] Convert pg_stat_database to new collector #685 +* [ENHANCEMENT] Supports alternate postgres:// prefix in URLs #787 +* [BUGFIX] Fix pg_setting different help values #771 +* [BUGFIX] Fix column type for pg_replication_slots #777 +* [BUGFIX] Fix pg_stat_database collector #809 + +## 0.12.1 / 2023-06-12 +* [BUGFIX] Fix column type for pg_replication_slots #777 + +## 0.12.0 / 2023-03-21 BREAKING CHANGES: @@ -10,7 +70,11 @@ PostgreSQL servers from a single exporter by passing the target via URL params. See the Multi-Target Support section of the README. * [CHANGE] Add multi-target support #618 +* [CHANGE] Add usename and application_name to pg_stat_activity metrics #673 +* [FEATURE] Add replication metrics from pg_replication_slots #747 * [BUGFIX] Add dsn type for handling datasources #678 +* [BUGFIX] Add 64kB unit for postgres 15 #740 +* [BUGFIX] Add 4kB unit for postgres compiled with small blocks #699 ## 0.11.1 / 2022-08-01 diff --git a/Makefile.common b/Makefile.common index 7642c4485..062a28185 100644 --- a/Makefile.common +++ b/Makefile.common @@ -49,19 +49,19 @@ endif GOTEST := $(GO) test GOTEST_DIR := ifneq ($(CIRCLE_JOB),) -ifneq ($(shell which gotestsum),) +ifneq ($(shell command -v gotestsum > /dev/null),) GOTEST_DIR := test-results GOTEST := gotestsum --junitfile $(GOTEST_DIR)/unit-tests.xml -- endif endif -PROMU_VERSION ?= 0.13.0 +PROMU_VERSION ?= 0.15.0 PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_VERSION)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM).tar.gz SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v1.49.0 +GOLANGCI_LINT_VERSION ?= v1.54.2 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64. # windows isn't included here because of the path separator being different. ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin)) @@ -91,6 +91,8 @@ BUILD_DOCKER_ARCHS = $(addprefix common-docker-,$(DOCKER_ARCHS)) PUBLISH_DOCKER_ARCHS = $(addprefix common-docker-publish-,$(DOCKER_ARCHS)) TAG_DOCKER_ARCHS = $(addprefix common-docker-tag-latest-,$(DOCKER_ARCHS)) +SANITIZED_DOCKER_IMAGE_TAG := $(subst +,-,$(DOCKER_IMAGE_TAG)) + ifeq ($(GOHOSTARCH),amd64) ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux freebsd darwin windows)) # Only supported on amd64 @@ -176,7 +178,7 @@ endif .PHONY: common-yamllint common-yamllint: @echo ">> running yamllint on all YAML files in the repository" -ifeq (, $(shell which yamllint)) +ifeq (, $(shell command -v yamllint > /dev/null)) @echo "yamllint not installed so skipping" else yamllint . @@ -205,7 +207,7 @@ common-tarball: promu .PHONY: common-docker $(BUILD_DOCKER_ARCHS) common-docker: $(BUILD_DOCKER_ARCHS) $(BUILD_DOCKER_ARCHS): common-docker-%: - docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(DOCKER_IMAGE_TAG)" \ + docker build -t "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" \ -f $(DOCKERFILE_PATH) \ --build-arg ARCH="$*" \ --build-arg OS="linux" \ @@ -214,19 +216,19 @@ $(BUILD_DOCKER_ARCHS): common-docker-%: .PHONY: common-docker-publish $(PUBLISH_DOCKER_ARCHS) common-docker-publish: $(PUBLISH_DOCKER_ARCHS) $(PUBLISH_DOCKER_ARCHS): common-docker-publish-%: - docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(DOCKER_IMAGE_TAG)" + docker push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" DOCKER_MAJOR_VERSION_TAG = $(firstword $(subst ., ,$(shell cat VERSION))) .PHONY: common-docker-tag-latest $(TAG_DOCKER_ARCHS) common-docker-tag-latest: $(TAG_DOCKER_ARCHS) $(TAG_DOCKER_ARCHS): common-docker-tag-latest-%: - docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest" - docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)" + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:latest" + docker tag "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:$(SANITIZED_DOCKER_IMAGE_TAG)" "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$*:v$(DOCKER_MAJOR_VERSION_TAG)" .PHONY: common-docker-manifest common-docker-manifest: - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(DOCKER_IMAGE_TAG)) - DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(DOCKER_IMAGE_TAG)" + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest create -a "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" $(foreach ARCH,$(DOCKER_ARCHS),$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME)-linux-$(ARCH):$(SANITIZED_DOCKER_IMAGE_TAG)) + DOCKER_CLI_EXPERIMENTAL=enabled docker manifest push "$(DOCKER_REPO)/$(DOCKER_IMAGE_NAME):$(SANITIZED_DOCKER_IMAGE_TAG)" .PHONY: promu promu: $(PROMU) diff --git a/README.md b/README.md index 2fc0ad8bc..429058e6d 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Prometheus exporter for PostgreSQL server metrics. -CI Tested PostgreSQL versions: `9.4`, `9.5`, `9.6`, `10`, `11`, `12`, `13`, `14` +CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16` ## Quick Start This package is available for Docker: @@ -30,6 +30,26 @@ To use the multi-target functionality, send an http request to the endpoint `/pr To avoid putting sensitive information like username and password in the URL, preconfigured auth modules are supported via the [auth_modules](#auth_modules) section of the config file. auth_modules for DSNs can be used with the `/probe` endpoint by specifying the `?auth_module=foo` http parameter. +Example Prometheus config: +```yaml +scrape_configs: + - job_name: 'postgres' + static_configs: + - targets: + - server1:5432 + - server2:5432 + metrics_path: /probe + params: + auth_module: [foo] + relabel_configs: + - source_labels: [__address__] + target_label: __param_target + - source_labels: [__param_target] + target_label: instance + - target_label: __address__ + replacement: 127.0.0.1:9116 # The postgres exporter's real hostname:port. +``` + ## Configuration File The configuration file controls the behavior of the exporter. It can be set using the `--config.file` command line flag and defaults to `postgres_exporter.yml`. @@ -61,7 +81,7 @@ auth_modules: To build the Docker image: make promu - promu crossbuild -p linux/amd64 -p linux/armv7 -p linux/amd64 -p linux/ppc64le + promu crossbuild -p linux/amd64 -p linux/armv7 -p linux/arm64 -p linux/ppc64le make docker This will build the docker image as `prometheuscommunity/postgres_exporter:${branch}`. @@ -71,15 +91,74 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `help` Show context-sensitive help (also try --help-long and --help-man). -* `collector.database` - Enable the pg_database collector. Default is `enabled` -* `collector.bgwriter` - Enable the pg_stat_bgwriter collector. Default is `enabled` +* `[no-]collector.database` + Enable the `database` collector (default: enabled). + +* `[no-]collector.database_wraparound` + Enable the `database_wraparound` collector (default: disabled). + +* `[no-]collector.locks` + Enable the `locks` collector (default: enabled). + +* `[no-]collector.long_running_transactions` + Enable the `long_running_transactions` collector (default: disabled). + +* `[no-]collector.postmaster` + Enable the `postmaster` collector (default: disabled). + +* `[no-]collector.process_idle` + Enable the `process_idle` collector (default: disabled). + +* `[no-]collector.replication` + Enable the `replication` collector (default: enabled). + +* `[no-]collector.replication_slot` + Enable the `replication_slot` collector (default: enabled). + +* `[no-]collector.stat_activity_autovacuum` + Enable the `stat_activity_autovacuum` collector (default: disabled). + +* `[no-]collector.stat_bgwriter` + Enable the `stat_bgwriter` collector (default: enabled). + +* `[no-]collector.stat_database` + Enable the `stat_database` collector (default: enabled). + +* `[no-]collector.stat_statements` + Enable the `stat_statements` collector (default: disabled). + +* `[no-]collector.stat_user_tables` + Enable the `stat_user_tables` collector (default: enabled). + +* `[no-]collector.stat_wal_receiver` + Enable the `stat_wal_receiver` collector (default: disabled). + +* `[no-]collector.statio_user_indexes` + Enable the `statio_user_indexes` collector (default: disabled). + +* `[no-]collector.statio_user_tables` + Enable the `statio_user_tables` collector (default: enabled). + +* `[no-]collector.wal` + Enable the `wal` collector (default: enabled). + +* `[no-]collector.xlog_location` + Enable the `xlog_location` collector (default: disabled). + +* `config.file` + Set the config file path. Default is `postgres_exporter.yml` + +* `web.systemd-socket` + Use systemd socket activation listeners instead of port listeners (Linux only). Default is `false` * `web.listen-address` Address to listen on for web interface and telemetry. Default is `:9187`. +* `web.config.file` + Configuration file to use TLS and/or basic authentication. The format of the + file is described [in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md). + * `web.telemetry-path` Path under which to expose metrics. Default is `/metrics`. @@ -89,10 +168,10 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `disable-settings-metrics` Use the flag if you don't want to scrape `pg_settings`. Default is `false`. -* `auto-discover-databases` +* `auto-discover-databases` (DEPRECATED) Whether to discover the databases on a server dynamically. Default is `false`. -* `extend.query-path` +* `extend.query-path` (DEPRECATED) Path to a YAML file containing custom queries to run. Check out [`queries.yaml`](queries.yaml) for examples of the format. @@ -100,16 +179,16 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra Do not run - print the internal representation of the metric maps. Useful when debugging a custom queries file. -* `constantLabels` +* `constantLabels` (DEPRECATED) Labels to set in all metrics. A list of `label=value` pairs, separated by commas. * `version` Show application version. -* `exclude-databases` +* `exclude-databases` (DEPRECATED) A list of databases to remove when autoDiscoverDatabases is enabled. -* `include-databases` +* `include-databases` (DEPRECATED) A list of databases to only include when autoDiscoverDatabases is enabled. * `log.level` @@ -118,10 +197,6 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra * `log.format` Set the log format: one of `logfmt`, `json`. -* `web.config.file` - Configuration file to use TLS and/or basic authentication. The format of the - file is described [in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md). - ### Environment Variables The following environment variables configure the exporter: @@ -152,9 +227,6 @@ The following environment variables configure the exporter: * `DATA_SOURCE_PASS_FILE` The same as above but reads the password from a file. -* `PG_EXPORTER_WEB_LISTEN_ADDRESS` - Address to listen on for web interface and telemetry. Default is `:9187`. - * `PG_EXPORTER_WEB_TELEMETRY_PATH` Path under which to expose metrics. Default is `/metrics`. @@ -164,20 +236,20 @@ The following environment variables configure the exporter: * `PG_EXPORTER_DISABLE_SETTINGS_METRICS` Use the flag if you don't want to scrape `pg_settings`. Value can be `true` or `false`. Default is `false`. -* `PG_EXPORTER_AUTO_DISCOVER_DATABASES` +* `PG_EXPORTER_AUTO_DISCOVER_DATABASES` (DEPRECATED) Whether to discover the databases on a server dynamically. Value can be `true` or `false`. Default is `false`. * `PG_EXPORTER_EXTEND_QUERY_PATH` Path to a YAML file containing custom queries to run. Check out [`queries.yaml`](queries.yaml) for examples of the format. -* `PG_EXPORTER_CONSTANT_LABELS` +* `PG_EXPORTER_CONSTANT_LABELS` (DEPRECATED) Labels to set in all metrics. A list of `label=value` pairs, separated by commas. -* `PG_EXPORTER_EXCLUDE_DATABASES` +* `PG_EXPORTER_EXCLUDE_DATABASES` (DEPRECATED) A comma-separated list of databases to remove when autoDiscoverDatabases is enabled. Default is empty string. -* `PG_EXPORTER_INCLUDE_DATABASES` +* `PG_EXPORTER_INCLUDE_DATABASES` (DEPRECATED) A comma-separated list of databases to only include when autoDiscoverDatabases is enabled. Default is empty string, means allow all. @@ -216,7 +288,9 @@ for l in StringIO(x): Adjust the value of the resultant prometheus value type appropriately. This helps build rich self-documenting metrics for the exporter. -### Adding new metrics via a config file +### Adding new metrics via a config file (DEPRECATED) + +This feature is deprecated in favor of built-in collector functions. For generic SQL database monitoring see the [sql_exporter](https://github.com/burningalchemist/sql_exporter). The -extend.query-path command-line argument specifies a YAML file containing additional queries to run. Some examples are provided in [queries.yaml](queries.yaml). @@ -227,7 +301,7 @@ or variants of postgres (e.g. Greenplum), you can disable the default metrics wi flag. This removes all built-in metrics, and uses only metrics defined by queries in the `queries.yaml` file you supply (so you must supply one, otherwise the exporter will return nothing but internal statuses and not your database). -### Automatically discover databases +### Automatically discover databases (DEPRECATED) To scrape metrics from all databases on a database server, the database DSN's can be dynamically discovered via the `--auto-discover-databases` flag. When true, `SELECT datname FROM pg_database WHERE datallowconn = true AND datistemplate = false and datname != current_database()` is run for all configured DSN's. From the result a new set of DSN's is created for which the metrics are scraped. diff --git a/VERSION b/VERSION index 68f8b7694..a55105169 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.12.0-rc.0 +0.15.0 diff --git a/cmd/postgres_exporter/datasource.go b/cmd/postgres_exporter/datasource.go index 97f7ecd8b..0b8cef04a 100644 --- a/cmd/postgres_exporter/datasource.go +++ b/cmd/postgres_exporter/datasource.go @@ -15,7 +15,6 @@ package main import ( "fmt" - "io/ioutil" "net/url" "os" "regexp" @@ -36,7 +35,7 @@ func (e *Exporter) discoverDatabaseDSNs() []string { var dsnURI *url.URL var dsnConnstring string - if strings.HasPrefix(dsn, "postgresql://") { + if strings.HasPrefix(dsn, "postgresql://") || strings.HasPrefix(dsn, "postgres://") { var err error dsnURI, err = url.Parse(dsn) if err != nil { @@ -130,7 +129,7 @@ func getDataSources() ([]string, error) { dataSourceUserFile := os.Getenv("DATA_SOURCE_USER_FILE") if len(dataSourceUserFile) != 0 { - fileContents, err := ioutil.ReadFile(dataSourceUserFile) + fileContents, err := os.ReadFile(dataSourceUserFile) if err != nil { return nil, fmt.Errorf("failed loading data source user file %s: %s", dataSourceUserFile, err.Error()) } @@ -141,7 +140,7 @@ func getDataSources() ([]string, error) { dataSourcePassFile := os.Getenv("DATA_SOURCE_PASS_FILE") if len(dataSourcePassFile) != 0 { - fileContents, err := ioutil.ReadFile(dataSourcePassFile) + fileContents, err := os.ReadFile(dataSourcePassFile) if err != nil { return nil, fmt.Errorf("failed loading data source pass file %s: %s", dataSourcePassFile, err.Error()) } @@ -153,7 +152,7 @@ func getDataSources() ([]string, error) { ui := url.UserPassword(user, pass).String() dataSrouceURIFile := os.Getenv("DATA_SOURCE_URI_FILE") if len(dataSrouceURIFile) != 0 { - fileContents, err := ioutil.ReadFile(dataSrouceURIFile) + fileContents, err := os.ReadFile(dataSrouceURIFile) if err != nil { return nil, fmt.Errorf("failed loading data source URI file %s: %s", dataSrouceURIFile, err.Error()) } diff --git a/cmd/postgres_exporter/main.go b/cmd/postgres_exporter/main.go index 8759f4372..f4d454996 100644 --- a/cmd/postgres_exporter/main.go +++ b/cmd/postgres_exporter/main.go @@ -14,9 +14,12 @@ package main import ( + "fmt" "net/http" "os" + "strings" + "github.com/alecthomas/kingpin/v2" "github.com/go-kit/log" "github.com/go-kit/log/level" "github.com/prometheus-community/postgres_exporter/collector" @@ -27,27 +30,25 @@ import ( "github.com/prometheus/common/promlog/flag" "github.com/prometheus/common/version" "github.com/prometheus/exporter-toolkit/web" - webflag "github.com/prometheus/exporter-toolkit/web/kingpinflag" - "gopkg.in/alecthomas/kingpin.v2" + "github.com/prometheus/exporter-toolkit/web/kingpinflag" ) var ( - c = config.ConfigHandler{ + c = config.Handler{ Config: &config.Config{}, } configFile = kingpin.Flag("config.file", "Postgres exporter configuration file.").Default("postgres_exporter.yml").String() - listenAddress = kingpin.Flag("web.listen-address", "Address to listen on for web interface and telemetry.").Default(":9187").Envar("PG_EXPORTER_WEB_LISTEN_ADDRESS").String() - webConfig = webflag.AddFlags(kingpin.CommandLine) - metricPath = kingpin.Flag("web.telemetry-path", "Path under which to expose metrics.").Default("/metrics").Envar("PG_EXPORTER_WEB_TELEMETRY_PATH").String() + webConfig = kingpinflag.AddFlags(kingpin.CommandLine, ":9187") + metricsPath = kingpin.Flag("web.telemetry-path", "Path under which to expose metrics.").Default("/metrics").Envar("PG_EXPORTER_WEB_TELEMETRY_PATH").String() disableDefaultMetrics = kingpin.Flag("disable-default-metrics", "Do not include default metrics.").Default("false").Envar("PG_EXPORTER_DISABLE_DEFAULT_METRICS").Bool() disableSettingsMetrics = kingpin.Flag("disable-settings-metrics", "Do not include pg_settings metrics.").Default("false").Envar("PG_EXPORTER_DISABLE_SETTINGS_METRICS").Bool() - autoDiscoverDatabases = kingpin.Flag("auto-discover-databases", "Whether to discover the databases on a server dynamically.").Default("false").Envar("PG_EXPORTER_AUTO_DISCOVER_DATABASES").Bool() - queriesPath = kingpin.Flag("extend.query-path", "Path to custom queries to run.").Default("").Envar("PG_EXPORTER_EXTEND_QUERY_PATH").String() + autoDiscoverDatabases = kingpin.Flag("auto-discover-databases", "Whether to discover the databases on a server dynamically. (DEPRECATED)").Default("false").Envar("PG_EXPORTER_AUTO_DISCOVER_DATABASES").Bool() + queriesPath = kingpin.Flag("extend.query-path", "Path to custom queries to run. (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXTEND_QUERY_PATH").String() onlyDumpMaps = kingpin.Flag("dumpmaps", "Do not run, simply dump the maps.").Bool() - constantLabelsList = kingpin.Flag("constantLabels", "A list of label=value separated by comma(,).").Default("").Envar("PG_EXPORTER_CONSTANT_LABELS").String() - excludeDatabases = kingpin.Flag("exclude-databases", "A list of databases to remove when autoDiscoverDatabases is enabled").Default("").Envar("PG_EXPORTER_EXCLUDE_DATABASES").String() - includeDatabases = kingpin.Flag("include-databases", "A list of databases to include when autoDiscoverDatabases is enabled").Default("").Envar("PG_EXPORTER_INCLUDE_DATABASES").String() + constantLabelsList = kingpin.Flag("constantLabels", "A list of label=value separated by comma(,). (DEPRECATED)").Default("").Envar("PG_EXPORTER_CONSTANT_LABELS").String() + excludeDatabases = kingpin.Flag("exclude-databases", "A list of databases to remove when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_EXCLUDE_DATABASES").String() + includeDatabases = kingpin.Flag("include-databases", "A list of databases to include when autoDiscoverDatabases is enabled (DEPRECATED)").Default("").Envar("PG_EXPORTER_INCLUDE_DATABASES").String() metricPrefix = kingpin.Flag("metric-prefix", "A metric prefix can be used to have non-default (not \"pg\") prefixes for each of the metrics").Default("pg").Envar("PG_EXPORTER_METRIC_PREFIX").String() logger = log.NewNopLogger() ) @@ -75,17 +76,6 @@ func main() { kingpin.Parse() logger = promlog.New(promlogConfig) - // landingPage contains the HTML served at '/'. - // TODO: Make this nicer and more informative. - var landingPage = []byte(` - Postgres exporter - -

Postgres exporter

-

Metrics

- - - `) - if *onlyDumpMaps { dumpMaps() return @@ -93,7 +83,7 @@ func main() { if err := c.ReloadConfig(*configFile, logger); err != nil { // This is not fatal, but it means that auth must be provided for every dsn. - level.Error(logger).Log("msg", "Error loading config", "err", err) + level.Warn(logger).Log("msg", "Error loading config", "err", err) } dsns, err := getDataSources() @@ -102,13 +92,28 @@ func main() { os.Exit(1) } + excludedDatabases := strings.Split(*excludeDatabases, ",") + level.Info(logger).Log("msg", "Excluded databases", "databases", fmt.Sprintf("%v", excludedDatabases)) + + if *queriesPath != "" { + level.Warn(logger).Log("msg", "The extended queries.yaml config is DEPRECATED", "file", *queriesPath) + } + + if *autoDiscoverDatabases || *excludeDatabases != "" || *includeDatabases != "" { + level.Warn(logger).Log("msg", "Scraping additional databases via auto discovery is DEPRECATED") + } + + if *constantLabelsList != "" { + level.Warn(logger).Log("msg", "Constant labels on all metrics is DEPRECATED") + } + opts := []ExporterOpt{ DisableDefaultMetrics(*disableDefaultMetrics), DisableSettingsMetrics(*disableSettingsMetrics), AutoDiscoverDatabases(*autoDiscoverDatabases), WithUserQueriesPath(*queriesPath), WithConstantLabels(*constantLabelsList), - ExcludeDatabases(*excludeDatabases), + ExcludeDatabases(excludedDatabases), IncludeDatabases(*includeDatabases), } @@ -129,26 +134,42 @@ func main() { pe, err := collector.NewPostgresCollector( logger, + excludedDatabases, dsn, []string{}, ) if err != nil { - level.Error(logger).Log("msg", "Failed to create PostgresCollector", "err", err.Error()) + level.Warn(logger).Log("msg", "Failed to create PostgresCollector", "err", err.Error()) } else { prometheus.MustRegister(pe) } - http.Handle(*metricPath, promhttp.Handler()) - http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/html; charset=UTF-8") // nolint: errcheck - w.Write(landingPage) // nolint: errcheck - }) + http.Handle(*metricsPath, promhttp.Handler()) + + if *metricsPath != "/" && *metricsPath != "" { + landingConfig := web.LandingConfig{ + Name: "Postgres Exporter", + Description: "Prometheus PostgreSQL server Exporter", + Version: version.Info(), + Links: []web.LandingLinks{ + { + Address: *metricsPath, + Text: "Metrics", + }, + }, + } + landingPage, err := web.NewLandingPage(landingConfig) + if err != nil { + level.Error(logger).Log("err", err) + os.Exit(1) + } + http.Handle("/", landingPage) + } - http.HandleFunc("/probe", handleProbe(logger)) + http.HandleFunc("/probe", handleProbe(logger, excludedDatabases)) - level.Info(logger).Log("msg", "Listening on address", "address", *listenAddress) - srv := &http.Server{Addr: *listenAddress} - if err := web.ListenAndServe(srv, *webConfig, logger); err != nil { + srv := &http.Server{} + if err := web.ListenAndServe(srv, webConfig, logger); err != nil { level.Error(logger).Log("msg", "Error running HTTP server", "err", err) os.Exit(1) } diff --git a/cmd/postgres_exporter/pg_setting.go b/cmd/postgres_exporter/pg_setting.go index 28a08a70b..f162c354a 100644 --- a/cmd/postgres_exporter/pg_setting.go +++ b/cmd/postgres_exporter/pg_setting.go @@ -70,7 +70,7 @@ func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric { err error name = strings.Replace(s.name, ".", "_", -1) unit = s.unit // nolint: ineffassign - shortDesc = s.shortDesc + shortDesc = fmt.Sprintf("Server Parameter: %s", s.name) subsystem = "settings" val float64 ) @@ -129,7 +129,7 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) { return case "ms", "s", "min", "h", "d": unit = "seconds" - case "B", "kB", "MB", "GB", "TB", "4kB", "8kB", "16kB", "32kB", "16MB", "32MB", "64MB": + case "B", "kB", "MB", "GB", "TB", "1kB", "2kB", "4kB", "8kB", "16kB", "32kB", "64kB", "16MB", "32MB", "64MB": unit = "bytes" default: err = fmt.Errorf("Unknown unit for runtime variable: %q", s.unit) @@ -158,6 +158,10 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) { val *= math.Pow(2, 30) case "TB": val *= math.Pow(2, 40) + case "1kB": + val *= math.Pow(2, 10) + case "2kB": + val *= math.Pow(2, 11) case "4kB": val *= math.Pow(2, 12) case "8kB": @@ -166,6 +170,8 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) { val *= math.Pow(2, 14) case "32kB": val *= math.Pow(2, 15) + case "64kB": + val *= math.Pow(2, 16) case "16MB": val *= math.Pow(2, 24) case "32MB": diff --git a/cmd/postgres_exporter/pg_setting_test.go b/cmd/postgres_exporter/pg_setting_test.go index 8f8f9c058..0e010444d 100644 --- a/cmd/postgres_exporter/pg_setting_test.go +++ b/cmd/postgres_exporter/pg_setting_test.go @@ -40,7 +40,7 @@ var fixtures = []fixture{ unit: "seconds", err: "", }, - d: `Desc{fqName: "pg_settings_seconds_fixture_metric_seconds", help: "Foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_seconds_fixture_metric_seconds", help: "Server Parameter: seconds_fixture_metric [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, v: 5, }, { @@ -56,7 +56,7 @@ var fixtures = []fixture{ unit: "seconds", err: "", }, - d: `Desc{fqName: "pg_settings_milliseconds_fixture_metric_seconds", help: "Foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_milliseconds_fixture_metric_seconds", help: "Server Parameter: milliseconds_fixture_metric [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, v: 5, }, { @@ -72,7 +72,7 @@ var fixtures = []fixture{ unit: "bytes", err: "", }, - d: `Desc{fqName: "pg_settings_eight_kb_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_eight_kb_fixture_metric_bytes", help: "Server Parameter: eight_kb_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, v: 139264, }, { @@ -88,7 +88,7 @@ var fixtures = []fixture{ unit: "bytes", err: "", }, - d: `Desc{fqName: "pg_settings_16_kb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_16_kb_real_fixture_metric_bytes", help: "Server Parameter: 16_kb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, v: 49152, }, { @@ -104,7 +104,7 @@ var fixtures = []fixture{ unit: "bytes", err: "", }, - d: `Desc{fqName: "pg_settings_16_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_16_mb_real_fixture_metric_bytes", help: "Server Parameter: 16_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, v: 5.0331648e+07, }, { @@ -120,7 +120,7 @@ var fixtures = []fixture{ unit: "bytes", err: "", }, - d: `Desc{fqName: "pg_settings_32_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_32_mb_real_fixture_metric_bytes", help: "Server Parameter: 32_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, v: 1.00663296e+08, }, { @@ -136,7 +136,7 @@ var fixtures = []fixture{ unit: "bytes", err: "", }, - d: `Desc{fqName: "pg_settings_64_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_64_mb_real_fixture_metric_bytes", help: "Server Parameter: 64_mb_real_fixture_metric [Units converted to bytes.]", constLabels: {}, variableLabels: {}}`, v: 2.01326592e+08, }, { @@ -152,7 +152,7 @@ var fixtures = []fixture{ unit: "", err: "", }, - d: `Desc{fqName: "pg_settings_bool_on_fixture_metric", help: "Foo foo foo", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_bool_on_fixture_metric", help: "Server Parameter: bool_on_fixture_metric", constLabels: {}, variableLabels: {}}`, v: 1, }, { @@ -168,7 +168,7 @@ var fixtures = []fixture{ unit: "", err: "", }, - d: `Desc{fqName: "pg_settings_bool_off_fixture_metric", help: "Foo foo foo", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_bool_off_fixture_metric", help: "Server Parameter: bool_off_fixture_metric", constLabels: {}, variableLabels: {}}`, v: 0, }, { @@ -184,7 +184,7 @@ var fixtures = []fixture{ unit: "seconds", err: "", }, - d: `Desc{fqName: "pg_settings_special_minus_one_value_seconds", help: "foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_special_minus_one_value_seconds", help: "Server Parameter: special_minus_one_value [Units converted to seconds.]", constLabels: {}, variableLabels: {}}`, v: -1, }, { @@ -200,7 +200,7 @@ var fixtures = []fixture{ unit: "", err: "", }, - d: `Desc{fqName: "pg_settings_rds_rds_superuser_reserved_connections", help: "Sets the number of connection slots reserved for rds_superusers.", constLabels: {}, variableLabels: []}`, + d: `Desc{fqName: "pg_settings_rds_rds_superuser_reserved_connections", help: "Server Parameter: rds.rds_superuser_reserved_connections", constLabels: {}, variableLabels: {}}`, v: 2, }, { diff --git a/cmd/postgres_exporter/postgres_exporter.go b/cmd/postgres_exporter/postgres_exporter.go index af973d5d5..fa34eecc5 100644 --- a/cmd/postgres_exporter/postgres_exporter.go +++ b/cmd/postgres_exporter/postgres_exporter.go @@ -18,8 +18,8 @@ import ( "database/sql" "errors" "fmt" - "io/ioutil" "math" + "os" "regexp" "strings" "time" @@ -163,31 +163,6 @@ func dumpMaps() { } var builtinMetricMaps = map[string]intermediateMetricMap{ - "pg_stat_database": { - map[string]ColumnMapping{ - "datid": {LABEL, "OID of a database", nil, nil}, - "datname": {LABEL, "Name of this database", nil, nil}, - "numbackends": {GAUGE, "Number of backends currently connected to this database. This is the only column in this view that returns a value reflecting current state; all other columns return the accumulated values since the last reset.", nil, nil}, - "xact_commit": {COUNTER, "Number of transactions in this database that have been committed", nil, nil}, - "xact_rollback": {COUNTER, "Number of transactions in this database that have been rolled back", nil, nil}, - "blks_read": {COUNTER, "Number of disk blocks read in this database", nil, nil}, - "blks_hit": {COUNTER, "Number of times disk blocks were found already in the buffer cache, so that a read was not necessary (this only includes hits in the PostgreSQL buffer cache, not the operating system's file system cache)", nil, nil}, - "tup_returned": {COUNTER, "Number of rows returned by queries in this database", nil, nil}, - "tup_fetched": {COUNTER, "Number of rows fetched by queries in this database", nil, nil}, - "tup_inserted": {COUNTER, "Number of rows inserted by queries in this database", nil, nil}, - "tup_updated": {COUNTER, "Number of rows updated by queries in this database", nil, nil}, - "tup_deleted": {COUNTER, "Number of rows deleted by queries in this database", nil, nil}, - "conflicts": {COUNTER, "Number of queries canceled due to conflicts with recovery in this database. (Conflicts occur only on standby servers; see pg_stat_database_conflicts for details.)", nil, nil}, - "temp_files": {COUNTER, "Number of temporary files created by queries in this database. All temporary files are counted, regardless of why the temporary file was created (e.g., sorting or hashing), and regardless of the log_temp_files setting.", nil, nil}, - "temp_bytes": {COUNTER, "Total amount of data written to temporary files by queries in this database. All temporary files are counted, regardless of why the temporary file was created, and regardless of the log_temp_files setting.", nil, nil}, - "deadlocks": {COUNTER, "Number of deadlocks detected in this database", nil, nil}, - "blk_read_time": {COUNTER, "Time spent reading data file blocks by backends in this database, in milliseconds", nil, nil}, - "blk_write_time": {COUNTER, "Time spent writing data file blocks by backends in this database, in milliseconds", nil, nil}, - "stats_reset": {COUNTER, "Time at which these statistics were last reset", nil, nil}, - }, - true, - 0, - }, "pg_stat_database_conflicts": { map[string]ColumnMapping{ "datid": {LABEL, "OID of a database", nil, nil}, @@ -201,26 +176,17 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ true, 0, }, - "pg_locks": { - map[string]ColumnMapping{ - "datname": {LABEL, "Name of this database", nil, nil}, - "mode": {LABEL, "Type of Lock", nil, nil}, - "count": {GAUGE, "Number of locks", nil, nil}, - }, - true, - 0, - }, "pg_stat_replication": { map[string]ColumnMapping{ - "procpid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange("<9.2.0")}, - "pid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange(">=9.2.0")}, - "usesysid": {DISCARD, "OID of the user logged into this WAL sender process", nil, nil}, - "usename": {DISCARD, "Name of the user logged into this WAL sender process", nil, nil}, - "application_name": {LABEL, "Name of the application that is connected to this WAL sender", nil, nil}, - "client_addr": {LABEL, "IP address of the client connected to this WAL sender. If this field is null, it indicates that the client is connected via a Unix socket on the server machine.", nil, nil}, - "client_hostname": {DISCARD, "Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when log_hostname is enabled.", nil, nil}, - "client_port": {DISCARD, "TCP port number that the client is using for communication with this WAL sender, or -1 if a Unix socket is used", nil, nil}, - "backend_start": {DISCARD, "with time zone Time when this process was started, i.e., when the client connected to this WAL sender", nil, nil}, + "procpid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange("<9.2.0")}, + "pid": {DISCARD, "Process ID of a WAL sender process", nil, semver.MustParseRange(">=9.2.0")}, + "usesysid": {DISCARD, "OID of the user logged into this WAL sender process", nil, nil}, + "usename": {DISCARD, "Name of the user logged into this WAL sender process", nil, nil}, + "application_name": {LABEL, "Name of the application that is connected to this WAL sender", nil, nil}, + "client_addr": {LABEL, "IP address of the client connected to this WAL sender. If this field is null, it indicates that the client is connected via a Unix socket on the server machine.", nil, nil}, + "client_hostname": {DISCARD, "Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when log_hostname is enabled.", nil, nil}, + "client_port": {DISCARD, "TCP port number that the client is using for communication with this WAL sender, or -1 if a Unix socket is used", nil, nil}, + "backend_start": {DISCARD, "with time zone Time when this process was started, i.e., when the client connected to this WAL sender", nil, nil}, "backend_xmin": {DISCARD, "The current backend's xmin horizon.", nil, nil}, "state": {LABEL, "Current WAL sender state", nil, nil}, "sent_location": {DISCARD, "Last transaction log position sent on this connection", nil, semver.MustParseRange("<10.0.0")}, @@ -282,10 +248,12 @@ var builtinMetricMaps = map[string]intermediateMetricMap{ }, "pg_stat_activity": { map[string]ColumnMapping{ - "datname": {LABEL, "Name of this database", nil, nil}, - "state": {LABEL, "connection state", nil, semver.MustParseRange(">=9.2.0")}, - "count": {GAUGE, "number of connections in this state", nil, nil}, - "max_tx_duration": {GAUGE, "max duration in seconds any active transaction has been running", nil, nil}, + "datname": {LABEL, "Name of this database", nil, nil}, + "state": {LABEL, "connection state", nil, semver.MustParseRange(">=9.2.0")}, + "usename": {LABEL, "connection usename", nil, nil}, + "application_name": {LABEL, "connection application_name", nil, nil}, + "count": {GAUGE, "number of connections in this state", nil, nil}, + "max_tx_duration": {GAUGE, "max duration in seconds any active transaction has been running", nil, nil}, }, true, 0, @@ -482,9 +450,9 @@ func AutoDiscoverDatabases(b bool) ExporterOpt { } // ExcludeDatabases allows to filter out result from AutoDiscoverDatabases -func ExcludeDatabases(s string) ExporterOpt { +func ExcludeDatabases(s []string) ExporterOpt { return func(e *Exporter) { - e.excludeDatabases = strings.Split(s, ",") + e.excludeDatabases = s } } @@ -661,7 +629,7 @@ func (e *Exporter) checkMapVersions(ch chan<- prometheus.Metric, server *Server) e.userQueriesError.Reset() // Calculate the hashsum of the useQueries - userQueriesData, err := ioutil.ReadFile(e.userQueriesPath) + userQueriesData, err := os.ReadFile(e.userQueriesPath) if err != nil { level.Error(logger).Log("msg", "Failed to reload user queries", "path", e.userQueriesPath, "err", err) e.userQueriesError.WithLabelValues(e.userQueriesPath, "").Set(1) diff --git a/cmd/postgres_exporter/postgres_exporter_test.go b/cmd/postgres_exporter/postgres_exporter_test.go index 468aa39b9..0f36febf4 100644 --- a/cmd/postgres_exporter/postgres_exporter_test.go +++ b/cmd/postgres_exporter/postgres_exporter_test.go @@ -17,7 +17,6 @@ package main import ( - "io/ioutil" "math" "os" "reflect" @@ -409,7 +408,7 @@ func (s *FunctionalSuite) TestBooleanConversionToValueAndString(c *C) { } func (s *FunctionalSuite) TestParseUserQueries(c *C) { - userQueriesData, err := ioutil.ReadFile("./tests/user_queries_ok.yaml") + userQueriesData, err := os.ReadFile("./tests/user_queries_ok.yaml") if err == nil { metricMaps, newQueryOverrides, err := parseUserQueries(userQueriesData) c.Assert(err, Equals, nil) diff --git a/cmd/postgres_exporter/probe.go b/cmd/postgres_exporter/probe.go index 28d1395ee..5945e07b8 100644 --- a/cmd/postgres_exporter/probe.go +++ b/cmd/postgres_exporter/probe.go @@ -16,7 +16,6 @@ package main import ( "fmt" "net/http" - "time" "github.com/go-kit/log" "github.com/go-kit/log/level" @@ -26,7 +25,7 @@ import ( "github.com/prometheus/client_golang/prometheus/promhttp" ) -func handleProbe(logger log.Logger) http.HandlerFunc { +func handleProbe(logger log.Logger, excludeDatabases []string) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() conf := c.GetConfig() @@ -62,21 +61,9 @@ func handleProbe(logger log.Logger) http.HandlerFunc { // TODO(@sysadmind): Timeout - probeSuccessGauge := prometheus.NewGauge(prometheus.GaugeOpts{ - Name: "probe_success", - Help: "Displays whether or not the probe was a success", - }) - probeDurationGauge := prometheus.NewGauge(prometheus.GaugeOpts{ - Name: "probe_duration_seconds", - Help: "Returns how long the probe took to complete in seconds", - }) - tl := log.With(logger, "target", target) - start := time.Now() registry := prometheus.NewRegistry() - registry.MustRegister(probeSuccessGauge) - registry.MustRegister(probeDurationGauge) opts := []ExporterOpt{ DisableDefaultMetrics(*disableDefaultMetrics), @@ -84,7 +71,7 @@ func handleProbe(logger log.Logger) http.HandlerFunc { AutoDiscoverDatabases(*autoDiscoverDatabases), WithUserQueriesPath(*queriesPath), WithConstantLabels(*constantLabelsList), - ExcludeDatabases(*excludeDatabases), + ExcludeDatabases(excludeDatabases), IncludeDatabases(*includeDatabases), } @@ -96,14 +83,16 @@ func handleProbe(logger log.Logger) http.HandlerFunc { registry.MustRegister(exporter) // Run the probe - pc, err := collector.NewProbeCollector(tl, registry, dsn) + pc, err := collector.NewProbeCollector(tl, excludeDatabases, registry, dsn) if err != nil { - probeSuccessGauge.Set(0) - probeDurationGauge.Set(time.Since(start).Seconds()) + level.Error(logger).Log("msg", "Error creating probe collector", "err", err) http.Error(w, err.Error(), http.StatusInternalServerError) return } + // Cleanup underlying connections to prevent connection leaks + defer pc.Close() + // TODO(@sysadmind): Remove the registry.MustRegister() call below and instead handle the collection here. That will allow // for the passing of context, handling of timeouts, and more control over the collection. // The current NewProbeCollector() implementation relies on the MustNewConstMetric() call to create the metrics which is not @@ -112,10 +101,6 @@ func handleProbe(logger log.Logger) http.HandlerFunc { registry.MustRegister(pc) - duration := time.Since(start).Seconds() - probeDurationGauge.Set(duration) - probeSuccessGauge.Set(1) - // TODO check success, etc h := promhttp.HandlerFor(registry, promhttp.HandlerOpts{}) h.ServeHTTP(w, r) diff --git a/cmd/postgres_exporter/queries.go b/cmd/postgres_exporter/queries.go index b17420d28..fa0b5c272 100644 --- a/cmd/postgres_exporter/queries.go +++ b/cmd/postgres_exporter/queries.go @@ -46,31 +46,6 @@ type OverrideQuery struct { // Overriding queries for namespaces above. // TODO: validate this is a closed set in tests, and there are no overlaps var queryOverrides = map[string][]OverrideQuery{ - "pg_locks": { - { - semver.MustParseRange(">0.0.0"), - `SELECT pg_database.datname,tmp.mode,COALESCE(count,0) as count - FROM - ( - VALUES ('accesssharelock'), - ('rowsharelock'), - ('rowexclusivelock'), - ('shareupdateexclusivelock'), - ('sharelock'), - ('sharerowexclusivelock'), - ('exclusivelock'), - ('accessexclusivelock'), - ('sireadlock') - ) AS tmp(mode) CROSS JOIN pg_database - LEFT JOIN - (SELECT database, lower(mode) AS mode,count(*) AS count - FROM pg_locks WHERE database IS NOT NULL - GROUP BY database, lower(mode) - ) AS tmp2 - ON tmp.mode=tmp2.mode and pg_database.oid = tmp2.database ORDER BY 1`, - }, - }, - "pg_stat_replication": { { semver.MustParseRange(">=10.0.0"), @@ -137,6 +112,8 @@ var queryOverrides = map[string][]OverrideQuery{ SELECT pg_database.datname, tmp.state, + tmp2.usename, + tmp2.application_name, COALESCE(count,0) as count, COALESCE(max_tx_duration,0) as max_tx_duration FROM @@ -153,9 +130,11 @@ var queryOverrides = map[string][]OverrideQuery{ SELECT datname, state, + usename, + application_name, count(*) AS count, MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration - FROM pg_stat_activity GROUP BY datname,state) AS tmp2 + FROM pg_stat_activity GROUP BY datname,state,usename,application_name) AS tmp2 ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname `, }, @@ -165,9 +144,11 @@ var queryOverrides = map[string][]OverrideQuery{ SELECT datname, 'unknown' AS state, + usename, + application_name, COALESCE(count(*),0) AS count, COALESCE(MAX(EXTRACT(EPOCH FROM now() - xact_start))::float,0) AS max_tx_duration - FROM pg_stat_activity GROUP BY datname + FROM pg_stat_activity GROUP BY datname,usename,application_name `, }, }, diff --git a/collector/collector.go b/collector/collector.go index 6d1a4dd11..121129871 100644 --- a/collector/collector.go +++ b/collector/collector.go @@ -15,20 +15,19 @@ package collector import ( "context" - "database/sql" "errors" "fmt" "sync" "time" + "github.com/alecthomas/kingpin/v2" "github.com/go-kit/log" "github.com/go-kit/log/level" "github.com/prometheus/client_golang/prometheus" - "gopkg.in/alecthomas/kingpin.v2" ) var ( - factories = make(map[string]func(logger log.Logger) (Collector, error)) + factories = make(map[string]func(collectorConfig) (Collector, error)) initiatedCollectorsMtx = sync.Mutex{} initiatedCollectors = make(map[string]Collector) collectorState = make(map[string]*bool) @@ -39,8 +38,8 @@ const ( // Namespace for all metrics. namespace = "pg" - defaultEnabled = true - // defaultDisabled = false + defaultEnabled = true + defaultDisabled = false ) var ( @@ -59,10 +58,15 @@ var ( ) type Collector interface { - Update(ctx context.Context, db *sql.DB, ch chan<- prometheus.Metric) error + Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error } -func registerCollector(name string, isDefaultEnabled bool, createFunc func(logger log.Logger) (Collector, error)) { +type collectorConfig struct { + logger log.Logger + excludeDatabases []string +} + +func registerCollector(name string, isDefaultEnabled bool, createFunc func(collectorConfig) (Collector, error)) { var helpDefaultState string if isDefaultEnabled { helpDefaultState = "enabled" @@ -87,13 +91,13 @@ type PostgresCollector struct { Collectors map[string]Collector logger log.Logger - db *sql.DB + instance *instance } type Option func(*PostgresCollector) error // NewPostgresCollector creates a new PostgresCollector. -func NewPostgresCollector(logger log.Logger, dsn string, filters []string, options ...Option) (*PostgresCollector, error) { +func NewPostgresCollector(logger log.Logger, excludeDatabases []string, dsn string, filters []string, options ...Option) (*PostgresCollector, error) { p := &PostgresCollector{ logger: logger, } @@ -126,7 +130,10 @@ func NewPostgresCollector(logger log.Logger, dsn string, filters []string, optio if collector, ok := initiatedCollectors[key]; ok { collectors[key] = collector } else { - collector, err := factories[key](log.With(logger, "collector", key)) + collector, err := factories[key](collectorConfig{ + logger: log.With(logger, "collector", key), + excludeDatabases: excludeDatabases, + }) if err != nil { return nil, err } @@ -141,14 +148,11 @@ func NewPostgresCollector(logger log.Logger, dsn string, filters []string, optio return nil, errors.New("empty dsn") } - db, err := sql.Open("postgres", dsn) + instance, err := newInstance(dsn) if err != nil { return nil, err } - db.SetMaxOpenConns(1) - db.SetMaxIdleConns(1) - - p.db = db + p.instance = instance return p, nil } @@ -162,20 +166,32 @@ func (p PostgresCollector) Describe(ch chan<- *prometheus.Desc) { // Collect implements the prometheus.Collector interface. func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) { ctx := context.TODO() + + // copy the instance so that concurrent scrapes have independent instances + inst := p.instance.copy() + + // Set up the database connection for the collector. + err := inst.setup() + if err != nil { + level.Error(p.logger).Log("msg", "Error opening connection to database", "err", err) + return + } + defer inst.Close() + wg := sync.WaitGroup{} wg.Add(len(p.Collectors)) for name, c := range p.Collectors { go func(name string, c Collector) { - execute(ctx, name, c, p.db, ch, p.logger) + execute(ctx, name, c, inst, ch, p.logger) wg.Done() }(name, c) } wg.Wait() } -func execute(ctx context.Context, name string, c Collector, db *sql.DB, ch chan<- prometheus.Metric, logger log.Logger) { +func execute(ctx context.Context, name string, c Collector, instance *instance, ch chan<- prometheus.Metric, logger log.Logger) { begin := time.Now() - err := c.Update(ctx, db, ch) + err := c.Update(ctx, instance, ch) duration := time.Since(begin) var success float64 diff --git a/collector/collector_test.go b/collector/collector_test.go new file mode 100644 index 000000000..18101f00e --- /dev/null +++ b/collector/collector_test.go @@ -0,0 +1,62 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "strings" + + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" +) + +type labelMap map[string]string + +type MetricResult struct { + labels labelMap + value float64 + metricType dto.MetricType +} + +func readMetric(m prometheus.Metric) MetricResult { + pb := &dto.Metric{} + m.Write(pb) + labels := make(labelMap, len(pb.Label)) + for _, v := range pb.Label { + labels[v.GetName()] = v.GetValue() + } + if pb.Gauge != nil { + return MetricResult{labels: labels, value: pb.GetGauge().GetValue(), metricType: dto.MetricType_GAUGE} + } + if pb.Counter != nil { + return MetricResult{labels: labels, value: pb.GetCounter().GetValue(), metricType: dto.MetricType_COUNTER} + } + if pb.Untyped != nil { + return MetricResult{labels: labels, value: pb.GetUntyped().GetValue(), metricType: dto.MetricType_UNTYPED} + } + panic("Unsupported metric type") +} + +func sanitizeQuery(q string) string { + q = strings.Join(strings.Fields(q), " ") + q = strings.Replace(q, "(", "\\(", -1) + q = strings.Replace(q, "?", "\\?", -1) + q = strings.Replace(q, ")", "\\)", -1) + q = strings.Replace(q, "[", "\\[", -1) + q = strings.Replace(q, "]", "\\]", -1) + q = strings.Replace(q, "{", "\\{", -1) + q = strings.Replace(q, "}", "\\}", -1) + q = strings.Replace(q, "*", "\\*", -1) + q = strings.Replace(q, "^", "\\^", -1) + q = strings.Replace(q, "$", "\\$", -1) + return q +} diff --git a/collector/instance.go b/collector/instance.go new file mode 100644 index 000000000..a365697d6 --- /dev/null +++ b/collector/instance.go @@ -0,0 +1,106 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "database/sql" + "fmt" + "regexp" + + "github.com/blang/semver/v4" +) + +type instance struct { + dsn string + db *sql.DB + version semver.Version +} + +func newInstance(dsn string) (*instance, error) { + i := &instance{ + dsn: dsn, + } + + // "Create" a database handle to verify the DSN provided is valid. + // Open is not guaranteed to create a connection. + db, err := sql.Open("postgres", dsn) + if err != nil { + return nil, err + } + db.Close() + + return i, nil +} + +// copy returns a copy of the instance. +func (i *instance) copy() *instance { + return &instance{ + dsn: i.dsn, + } +} + +func (i *instance) setup() error { + db, err := sql.Open("postgres", i.dsn) + if err != nil { + return err + } + db.SetMaxOpenConns(1) + db.SetMaxIdleConns(1) + i.db = db + + version, err := queryVersion(i.db) + if err != nil { + return fmt.Errorf("error querying postgresql version: %w", err) + } else { + i.version = version + } + return nil +} + +func (i *instance) getDB() *sql.DB { + return i.db +} + +func (i *instance) Close() error { + return i.db.Close() +} + +// Regex used to get the "short-version" from the postgres version field. +// The result of SELECT version() is something like "PostgreSQL 9.6.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 6.2.1 20160830, 64-bit" +var versionRegex = regexp.MustCompile(`^\w+ ((\d+)(\.\d+)?(\.\d+)?)`) +var serverVersionRegex = regexp.MustCompile(`^((\d+)(\.\d+)?(\.\d+)?)`) + +func queryVersion(db *sql.DB) (semver.Version, error) { + var version string + err := db.QueryRow("SELECT version();").Scan(&version) + if err != nil { + return semver.Version{}, err + } + submatches := versionRegex.FindStringSubmatch(version) + if len(submatches) > 1 { + return semver.ParseTolerant(submatches[1]) + } + + // We could also try to parse the version from the server_version field. + // This is of the format 13.3 (Debian 13.3-1.pgdg100+1) + err = db.QueryRow("SHOW server_version;").Scan(&version) + if err != nil { + return semver.Version{}, err + } + submatches = serverVersionRegex.FindStringSubmatch(version) + if len(submatches) > 1 { + return semver.ParseTolerant(submatches[1]) + } + return semver.Version{}, fmt.Errorf("could not parse version from %q", version) +} diff --git a/collector/pg_database.go b/collector/pg_database.go index 8fa4dab8e..d2c4b206a 100644 --- a/collector/pg_database.go +++ b/collector/pg_database.go @@ -21,50 +21,108 @@ import ( "github.com/prometheus/client_golang/prometheus" ) +const databaseSubsystem = "database" + func init() { - registerCollector("database", defaultEnabled, NewPGDatabaseCollector) + registerCollector(databaseSubsystem, defaultEnabled, NewPGDatabaseCollector) } type PGDatabaseCollector struct { - log log.Logger + log log.Logger + excludedDatabases []string } -func NewPGDatabaseCollector(logger log.Logger) (Collector, error) { - return &PGDatabaseCollector{log: logger}, nil +func NewPGDatabaseCollector(config collectorConfig) (Collector, error) { + exclude := config.excludeDatabases + if exclude == nil { + exclude = []string{} + } + return &PGDatabaseCollector{ + log: config.logger, + excludedDatabases: exclude, + }, nil } -var pgDatabase = map[string]*prometheus.Desc{ - "size_bytes": prometheus.NewDesc( - "pg_database_size_bytes", +var ( + pgDatabaseSizeDesc = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + databaseSubsystem, + "size_bytes", + ), "Disk space used by the database", []string{"datname"}, nil, - ), -} + ) + + pgDatabaseQuery = "SELECT pg_database.datname FROM pg_database;" + pgDatabaseSizeQuery = "SELECT pg_database_size($1)" +) -func (PGDatabaseCollector) Update(ctx context.Context, db *sql.DB, ch chan<- prometheus.Metric) error { +// Update implements Collector and exposes database size. +// It is called by the Prometheus registry when collecting metrics. +// The list of databases is retrieved from pg_database and filtered +// by the excludeDatabase config parameter. The tradeoff here is that +// we have to query the list of databases and then query the size of +// each database individually. This is because we can't filter the +// list of databases in the query because the list of excluded +// databases is dynamic. +func (c PGDatabaseCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + // Query the list of databases rows, err := db.QueryContext(ctx, - `SELECT pg_database.datname - ,pg_database_size(pg_database.datname) - FROM pg_database;`) + pgDatabaseQuery, + ) if err != nil { return err } defer rows.Close() + var databases []string + for rows.Next() { - var datname string - var size int64 - if err := rows.Scan(&datname, &size); err != nil { + var datname sql.NullString + if err := rows.Scan(&datname); err != nil { return err } + if !datname.Valid { + continue + } + // Ignore excluded databases + // Filtering is done here instead of in the query to avoid + // a complicated NOT IN query with a variable number of parameters + if sliceContains(c.excludedDatabases, datname.String) { + continue + } + + databases = append(databases, datname.String) + } + + // Query the size of the databases + for _, datname := range databases { + var size sql.NullFloat64 + err = db.QueryRowContext(ctx, pgDatabaseSizeQuery, datname).Scan(&size) + if err != nil { + return err + } + + sizeMetric := 0.0 + if size.Valid { + sizeMetric = size.Float64 + } ch <- prometheus.MustNewConstMetric( - pgDatabase["size_bytes"], - prometheus.GaugeValue, float64(size), datname, + pgDatabaseSizeDesc, + prometheus.GaugeValue, sizeMetric, datname, ) } - if err := rows.Err(); err != nil { - return err + return rows.Err() +} + +func sliceContains(slice []string, s string) bool { + for _, item := range slice { + if item == s { + return true + } } - return nil + return false } diff --git a/collector/pg_database_test.go b/collector/pg_database_test.go new file mode 100644 index 000000000..b5052c5d1 --- /dev/null +++ b/collector/pg_database_test.go @@ -0,0 +1,101 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGDatabaseCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + mock.ExpectQuery(sanitizeQuery(pgDatabaseQuery)).WillReturnRows(sqlmock.NewRows([]string{"datname"}). + AddRow("postgres")) + + mock.ExpectQuery(sanitizeQuery(pgDatabaseSizeQuery)).WithArgs("postgres").WillReturnRows(sqlmock.NewRows([]string{"pg_database_size"}). + AddRow(1024)) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGDatabaseCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres"}, value: 1024, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +// TODO add a null db test + +func TestPGDatabaseCollectorNullMetric(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + mock.ExpectQuery(sanitizeQuery(pgDatabaseQuery)).WillReturnRows(sqlmock.NewRows([]string{"datname"}). + AddRow("postgres")) + + mock.ExpectQuery(sanitizeQuery(pgDatabaseSizeQuery)).WithArgs("postgres").WillReturnRows(sqlmock.NewRows([]string{"pg_database_size"}). + AddRow(nil)) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGDatabaseCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres"}, value: 0, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_database_wraparound.go b/collector/pg_database_wraparound.go new file mode 100644 index 000000000..d46270637 --- /dev/null +++ b/collector/pg_database_wraparound.go @@ -0,0 +1,115 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/go-kit/log/level" + "github.com/prometheus/client_golang/prometheus" +) + +const databaseWraparoundSubsystem = "database_wraparound" + +func init() { + registerCollector(databaseWraparoundSubsystem, defaultDisabled, NewPGDatabaseWraparoundCollector) +} + +type PGDatabaseWraparoundCollector struct { + log log.Logger +} + +func NewPGDatabaseWraparoundCollector(config collectorConfig) (Collector, error) { + return &PGDatabaseWraparoundCollector{log: config.logger}, nil +} + +var ( + databaseWraparoundAgeDatfrozenxid = prometheus.NewDesc( + prometheus.BuildFQName(namespace, databaseWraparoundSubsystem, "age_datfrozenxid_seconds"), + "Age of the oldest transaction ID that has not been frozen.", + []string{"datname"}, + prometheus.Labels{}, + ) + databaseWraparoundAgeDatminmxid = prometheus.NewDesc( + prometheus.BuildFQName(namespace, databaseWraparoundSubsystem, "age_datminmxid_seconds"), + "Age of the oldest multi-transaction ID that has been replaced with a transaction ID.", + []string{"datname"}, + prometheus.Labels{}, + ) + + databaseWraparoundQuery = ` + SELECT + datname, + age(d.datfrozenxid) as age_datfrozenxid, + mxid_age(d.datminmxid) as age_datminmxid + FROM + pg_catalog.pg_database d + WHERE + d.datallowconn + ` +) + +func (c *PGDatabaseWraparoundCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + databaseWraparoundQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var datname sql.NullString + var ageDatfrozenxid, ageDatminmxid sql.NullFloat64 + + if err := rows.Scan(&datname, &ageDatfrozenxid, &ageDatminmxid); err != nil { + return err + } + + if !datname.Valid { + level.Debug(c.log).Log("msg", "Skipping database with NULL name") + continue + } + if !ageDatfrozenxid.Valid { + level.Debug(c.log).Log("msg", "Skipping stat emission with NULL age_datfrozenxid") + continue + } + if !ageDatminmxid.Valid { + level.Debug(c.log).Log("msg", "Skipping stat emission with NULL age_datminmxid") + continue + } + + ageDatfrozenxidMetric := ageDatfrozenxid.Float64 + + ch <- prometheus.MustNewConstMetric( + databaseWraparoundAgeDatfrozenxid, + prometheus.GaugeValue, + ageDatfrozenxidMetric, datname.String, + ) + + ageDatminmxidMetric := ageDatminmxid.Float64 + ch <- prometheus.MustNewConstMetric( + databaseWraparoundAgeDatminmxid, + prometheus.GaugeValue, + ageDatminmxidMetric, datname.String, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_database_wraparound_test.go b/collector/pg_database_wraparound_test.go new file mode 100644 index 000000000..d0a74c362 --- /dev/null +++ b/collector/pg_database_wraparound_test.go @@ -0,0 +1,64 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGDatabaseWraparoundCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "datname", + "age_datfrozenxid", + "age_datminmxid", + } + rows := sqlmock.NewRows(columns). + AddRow("newreddit", 87126426, 0) + + mock.ExpectQuery(sanitizeQuery(databaseWraparoundQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGDatabaseWraparoundCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGDatabaseWraparoundCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"datname": "newreddit"}, value: 87126426, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"datname": "newreddit"}, value: 0, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_locks.go b/collector/pg_locks.go new file mode 100644 index 000000000..d2c77ccd5 --- /dev/null +++ b/collector/pg_locks.go @@ -0,0 +1,129 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const locksSubsystem = "locks" + +func init() { + registerCollector(locksSubsystem, defaultEnabled, NewPGLocksCollector) +} + +type PGLocksCollector struct { + log log.Logger +} + +func NewPGLocksCollector(config collectorConfig) (Collector, error) { + return &PGLocksCollector{ + log: config.logger, + }, nil +} + +var ( + pgLocksDesc = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + locksSubsystem, + "count", + ), + "Number of locks", + []string{"datname", "mode"}, nil, + ) + + pgLocksQuery = ` + SELECT + pg_database.datname as datname, + tmp.mode as mode, + COALESCE(count, 0) as count + FROM + ( + VALUES + ('accesssharelock'), + ('rowsharelock'), + ('rowexclusivelock'), + ('shareupdateexclusivelock'), + ('sharelock'), + ('sharerowexclusivelock'), + ('exclusivelock'), + ('accessexclusivelock'), + ('sireadlock') + ) AS tmp(mode) + CROSS JOIN pg_database + LEFT JOIN ( + SELECT + database, + lower(mode) AS mode, + count(*) AS count + FROM + pg_locks + WHERE + database IS NOT NULL + GROUP BY + database, + lower(mode) + ) AS tmp2 ON tmp.mode = tmp2.mode + and pg_database.oid = tmp2.database + ORDER BY + 1 + ` +) + +// Update implements Collector and exposes database locks. +// It is called by the Prometheus registry when collecting metrics. +func (c PGLocksCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + // Query the list of databases + rows, err := db.QueryContext(ctx, + pgLocksQuery, + ) + if err != nil { + return err + } + defer rows.Close() + + var datname, mode sql.NullString + var count sql.NullInt64 + + for rows.Next() { + if err := rows.Scan(&datname, &mode, &count); err != nil { + return err + } + + if !datname.Valid || !mode.Valid { + continue + } + + countMetric := 0.0 + if count.Valid { + countMetric = float64(count.Int64) + } + + ch <- prometheus.MustNewConstMetric( + pgLocksDesc, + prometheus.GaugeValue, countMetric, + datname.String, mode.String, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_locks_test.go b/collector/pg_locks_test.go new file mode 100644 index 000000000..99597ea2d --- /dev/null +++ b/collector/pg_locks_test.go @@ -0,0 +1,60 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGLocksCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + rows := sqlmock.NewRows([]string{"datname", "mode", "count"}). + AddRow("test", "exclusivelock", 42) + + mock.ExpectQuery(sanitizeQuery(pgLocksQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGLocksCollector{} + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGLocksCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "test", "mode": "exclusivelock"}, value: 42, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_long_running_transactions.go b/collector/pg_long_running_transactions.go new file mode 100644 index 000000000..ffd89d5f0 --- /dev/null +++ b/collector/pg_long_running_transactions.go @@ -0,0 +1,93 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const longRunningTransactionsSubsystem = "long_running_transactions" + +func init() { + registerCollector(longRunningTransactionsSubsystem, defaultDisabled, NewPGLongRunningTransactionsCollector) +} + +type PGLongRunningTransactionsCollector struct { + log log.Logger +} + +func NewPGLongRunningTransactionsCollector(config collectorConfig) (Collector, error) { + return &PGLongRunningTransactionsCollector{log: config.logger}, nil +} + +var ( + longRunningTransactionsCount = prometheus.NewDesc( + "pg_long_running_transactions", + "Current number of long running transactions", + []string{}, + prometheus.Labels{}, + ) + + longRunningTransactionsAgeInSeconds = prometheus.NewDesc( + prometheus.BuildFQName(namespace, longRunningTransactionsSubsystem, "oldest_timestamp_seconds"), + "The current maximum transaction age in seconds", + []string{}, + prometheus.Labels{}, + ) + + longRunningTransactionsQuery = ` + SELECT + COUNT(*) as transactions, + MAX(EXTRACT(EPOCH FROM clock_timestamp())) AS oldest_timestamp_seconds + FROM pg_catalog.pg_stat_activity + WHERE state is distinct from 'idle' AND query not like 'autovacuum:%' + ` +) + +func (PGLongRunningTransactionsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + longRunningTransactionsQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var transactions, ageInSeconds float64 + + if err := rows.Scan(&transactions, &ageInSeconds); err != nil { + return err + } + + ch <- prometheus.MustNewConstMetric( + longRunningTransactionsCount, + prometheus.GaugeValue, + transactions, + ) + ch <- prometheus.MustNewConstMetric( + longRunningTransactionsAgeInSeconds, + prometheus.GaugeValue, + ageInSeconds, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_long_running_transactions_test.go b/collector/pg_long_running_transactions_test.go new file mode 100644 index 000000000..eedda7c65 --- /dev/null +++ b/collector/pg_long_running_transactions_test.go @@ -0,0 +1,63 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGLongRunningTransactionsCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "transactions", + "age_in_seconds", + } + rows := sqlmock.NewRows(columns). + AddRow(20, 1200) + + mock.ExpectQuery(sanitizeQuery(longRunningTransactionsQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGLongRunningTransactionsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGLongRunningTransactionsCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{}, value: 20, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{}, value: 1200, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_postmaster.go b/collector/pg_postmaster.go new file mode 100644 index 000000000..b81e4f905 --- /dev/null +++ b/collector/pg_postmaster.go @@ -0,0 +1,69 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/prometheus/client_golang/prometheus" +) + +const postmasterSubsystem = "postmaster" + +func init() { + registerCollector(postmasterSubsystem, defaultDisabled, NewPGPostmasterCollector) +} + +type PGPostmasterCollector struct { +} + +func NewPGPostmasterCollector(collectorConfig) (Collector, error) { + return &PGPostmasterCollector{}, nil +} + +var ( + pgPostMasterStartTimeSeconds = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + postmasterSubsystem, + "start_time_seconds", + ), + "Time at which postmaster started", + []string{}, nil, + ) + + pgPostmasterQuery = "SELECT extract(epoch from pg_postmaster_start_time) from pg_postmaster_start_time();" +) + +func (c *PGPostmasterCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + row := db.QueryRowContext(ctx, + pgPostmasterQuery) + + var startTimeSeconds sql.NullFloat64 + err := row.Scan(&startTimeSeconds) + if err != nil { + return err + } + startTimeSecondsMetric := 0.0 + if startTimeSeconds.Valid { + startTimeSecondsMetric = startTimeSeconds.Float64 + } + ch <- prometheus.MustNewConstMetric( + pgPostMasterStartTimeSeconds, + prometheus.GaugeValue, startTimeSecondsMetric, + ) + return nil +} diff --git a/collector/pg_postmaster_test.go b/collector/pg_postmaster_test.go new file mode 100644 index 000000000..8405b4225 --- /dev/null +++ b/collector/pg_postmaster_test.go @@ -0,0 +1,95 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPgPostmasterCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + mock.ExpectQuery(sanitizeQuery(pgPostmasterQuery)).WillReturnRows(sqlmock.NewRows([]string{"pg_postmaster_start_time"}). + AddRow(1685739904)) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGPostmasterCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGPostmasterCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, value: 1685739904, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPgPostmasterCollectorNullTime(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + mock.ExpectQuery(sanitizeQuery(pgPostmasterQuery)).WillReturnRows(sqlmock.NewRows([]string{"pg_postmaster_start_time"}). + AddRow(nil)) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGPostmasterCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGPostmasterCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, value: 0, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_process_idle.go b/collector/pg_process_idle.go new file mode 100644 index 000000000..c401ab56f --- /dev/null +++ b/collector/pg_process_idle.go @@ -0,0 +1,132 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/lib/pq" + "github.com/prometheus/client_golang/prometheus" +) + +func init() { + // Making this default disabled because we have no tests for it + registerCollector(processIdleSubsystem, defaultDisabled, NewPGProcessIdleCollector) +} + +type PGProcessIdleCollector struct { + log log.Logger +} + +const processIdleSubsystem = "process_idle" + +func NewPGProcessIdleCollector(config collectorConfig) (Collector, error) { + return &PGProcessIdleCollector{log: config.logger}, nil +} + +var pgProcessIdleSeconds = prometheus.NewDesc( + prometheus.BuildFQName(namespace, processIdleSubsystem, "seconds"), + "Idle time of server processes", + []string{"state", "application_name"}, + prometheus.Labels{}, +) + +func (PGProcessIdleCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + row := db.QueryRowContext(ctx, + `WITH + metrics AS ( + SELECT + state, + application_name, + SUM(EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP - state_change))::bigint)::float AS process_idle_seconds_sum, + COUNT(*) AS process_idle_seconds_count + FROM pg_stat_activity + WHERE state ~ '^idle' + GROUP BY state, application_name + ), + buckets AS ( + SELECT + state, + application_name, + le, + SUM( + CASE WHEN EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP - state_change)) <= le + THEN 1 + ELSE 0 + END + )::bigint AS bucket + FROM + pg_stat_activity, + UNNEST(ARRAY[1, 2, 5, 15, 30, 60, 90, 120, 300]) AS le + GROUP BY state, application_name, le + ORDER BY state, application_name, le + ) + SELECT + state, + application_name, + process_idle_seconds_sum as seconds_sum, + process_idle_seconds_count as seconds_count, + ARRAY_AGG(le) AS seconds, + ARRAY_AGG(bucket) AS seconds_bucket + FROM metrics JOIN buckets USING (state, application_name) + GROUP BY 1, 2, 3, 4;`) + + var state sql.NullString + var applicationName sql.NullString + var secondsSum sql.NullFloat64 + var secondsCount sql.NullInt64 + var seconds []float64 + var secondsBucket []int64 + + err := row.Scan(&state, &applicationName, &secondsSum, &secondsCount, pq.Array(&seconds), pq.Array(&secondsBucket)) + if err != nil { + return err + } + + var buckets = make(map[float64]uint64, len(seconds)) + for i, second := range seconds { + if i >= len(secondsBucket) { + break + } + buckets[second] = uint64(secondsBucket[i]) + } + + stateLabel := "unknown" + if state.Valid { + stateLabel = state.String + } + + applicationNameLabel := "unknown" + if applicationName.Valid { + applicationNameLabel = applicationName.String + } + + var secondsCountMetric uint64 + if secondsCount.Valid { + secondsCountMetric = uint64(secondsCount.Int64) + } + secondsSumMetric := 0.0 + if secondsSum.Valid { + secondsSumMetric = secondsSum.Float64 + } + ch <- prometheus.MustNewConstHistogram( + pgProcessIdleSeconds, + secondsCountMetric, secondsSumMetric, buckets, + stateLabel, applicationNameLabel, + ) + return nil +} diff --git a/collector/pg_replication.go b/collector/pg_replication.go new file mode 100644 index 000000000..6067cc9b1 --- /dev/null +++ b/collector/pg_replication.go @@ -0,0 +1,88 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + + "github.com/prometheus/client_golang/prometheus" +) + +const replicationSubsystem = "replication" + +func init() { + registerCollector(replicationSubsystem, defaultEnabled, NewPGReplicationCollector) +} + +type PGReplicationCollector struct { +} + +func NewPGReplicationCollector(collectorConfig) (Collector, error) { + return &PGReplicationCollector{}, nil +} + +var ( + pgReplicationLag = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSubsystem, + "lag_seconds", + ), + "Replication lag behind master in seconds", + []string{}, nil, + ) + pgReplicationIsReplica = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSubsystem, + "is_replica", + ), + "Indicates if the server is a replica", + []string{}, nil, + ) + + pgReplicationQuery = `SELECT + CASE + WHEN NOT pg_is_in_recovery() THEN 0 + WHEN pg_last_wal_receive_lsn () = pg_last_wal_replay_lsn () THEN 0 + ELSE GREATEST (0, EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) + END AS lag, + CASE + WHEN pg_is_in_recovery() THEN 1 + ELSE 0 + END as is_replica` +) + +func (c *PGReplicationCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + row := db.QueryRowContext(ctx, + pgReplicationQuery, + ) + + var lag float64 + var isReplica int64 + err := row.Scan(&lag, &isReplica) + if err != nil { + return err + } + ch <- prometheus.MustNewConstMetric( + pgReplicationLag, + prometheus.GaugeValue, lag, + ) + ch <- prometheus.MustNewConstMetric( + pgReplicationIsReplica, + prometheus.GaugeValue, float64(isReplica), + ) + return nil +} diff --git a/collector/pg_replication_slot.go b/collector/pg_replication_slot.go new file mode 100644 index 000000000..7f1ba003e --- /dev/null +++ b/collector/pg_replication_slot.go @@ -0,0 +1,136 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const replicationSlotSubsystem = "replication_slot" + +func init() { + registerCollector(replicationSlotSubsystem, defaultEnabled, NewPGReplicationSlotCollector) +} + +type PGReplicationSlotCollector struct { + log log.Logger +} + +func NewPGReplicationSlotCollector(config collectorConfig) (Collector, error) { + return &PGReplicationSlotCollector{log: config.logger}, nil +} + +var ( + pgReplicationSlotCurrentWalDesc = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSlotSubsystem, + "slot_current_wal_lsn", + ), + "current wal lsn value", + []string{"slot_name", "slot_type"}, nil, + ) + pgReplicationSlotCurrentFlushDesc = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSlotSubsystem, + "slot_confirmed_flush_lsn", + ), + "last lsn confirmed flushed to the replication slot", + []string{"slot_name", "slot_type"}, nil, + ) + pgReplicationSlotIsActiveDesc = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + replicationSlotSubsystem, + "slot_is_active", + ), + "whether the replication slot is active or not", + []string{"slot_name", "slot_type"}, nil, + ) + + pgReplicationSlotQuery = `SELECT + slot_name, + slot_type, + CASE WHEN pg_is_in_recovery() THEN + pg_last_wal_receive_lsn() - '0/0' + ELSE + pg_current_wal_lsn() - '0/0' + END AS current_wal_lsn, + COALESCE(confirmed_flush_lsn, '0/0') - '0/0' AS confirmed_flush_lsn, + active + FROM pg_replication_slots;` +) + +func (PGReplicationSlotCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + pgReplicationSlotQuery) + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var slotName sql.NullString + var slotType sql.NullString + var walLSN sql.NullFloat64 + var flushLSN sql.NullFloat64 + var isActive sql.NullBool + if err := rows.Scan(&slotName, &slotType, &walLSN, &flushLSN, &isActive); err != nil { + return err + } + + isActiveValue := 0.0 + if isActive.Valid && isActive.Bool { + isActiveValue = 1.0 + } + slotNameLabel := "unknown" + if slotName.Valid { + slotNameLabel = slotName.String + } + slotTypeLabel := "unknown" + if slotType.Valid { + slotTypeLabel = slotType.String + } + + var walLSNMetric float64 + if walLSN.Valid { + walLSNMetric = walLSN.Float64 + } + ch <- prometheus.MustNewConstMetric( + pgReplicationSlotCurrentWalDesc, + prometheus.GaugeValue, walLSNMetric, slotNameLabel, slotTypeLabel, + ) + if isActive.Valid && isActive.Bool { + var flushLSNMetric float64 + if flushLSN.Valid { + flushLSNMetric = flushLSN.Float64 + } + ch <- prometheus.MustNewConstMetric( + pgReplicationSlotCurrentFlushDesc, + prometheus.GaugeValue, flushLSNMetric, slotNameLabel, slotTypeLabel, + ) + } + ch <- prometheus.MustNewConstMetric( + pgReplicationSlotIsActiveDesc, + prometheus.GaugeValue, isActiveValue, slotNameLabel, slotTypeLabel, + ) + } + return rows.Err() +} diff --git a/collector/pg_replication_slot_test.go b/collector/pg_replication_slot_test.go new file mode 100644 index 000000000..212050c46 --- /dev/null +++ b/collector/pg_replication_slot_test.go @@ -0,0 +1,186 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPgReplicationSlotCollectorActive(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active"} + rows := sqlmock.NewRows(columns). + AddRow("test_slot", "physical", 5, 3, true) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGReplicationSlotCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGPostmasterCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 5, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 3, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 1, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPgReplicationSlotCollectorInActive(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active"} + rows := sqlmock.NewRows(columns). + AddRow("test_slot", "physical", 6, 12, false) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGReplicationSlotCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGReplicationSlotCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 6, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 0, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } + +} + +func TestPgReplicationSlotCollectorActiveNil(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active"} + rows := sqlmock.NewRows(columns). + AddRow("test_slot", "physical", 6, 12, nil) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGReplicationSlotCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGReplicationSlotCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 6, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "test_slot", "slot_type": "physical"}, value: 0, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPgReplicationSlotCollectorTestNilValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"slot_name", "slot_type", "current_wal_lsn", "confirmed_flush_lsn", "active"} + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, true) + mock.ExpectQuery(sanitizeQuery(pgReplicationSlotQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGReplicationSlotCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGReplicationSlotCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"slot_name": "unknown", "slot_type": "unknown"}, value: 0, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "unknown", "slot_type": "unknown"}, value: 0, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"slot_name": "unknown", "slot_type": "unknown"}, value: 1, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_replication_test.go b/collector/pg_replication_test.go new file mode 100644 index 000000000..b6df698e3 --- /dev/null +++ b/collector/pg_replication_test.go @@ -0,0 +1,63 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPgReplicationCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"lag", "is_replica"} + rows := sqlmock.NewRows(columns). + AddRow(1000, 1) + mock.ExpectQuery(sanitizeQuery(pgReplicationQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGReplicationCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGReplicationCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, value: 1000, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{}, value: 1, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_activity_autovacuum.go b/collector/pg_stat_activity_autovacuum.go new file mode 100644 index 000000000..6cf8cdcec --- /dev/null +++ b/collector/pg_stat_activity_autovacuum.go @@ -0,0 +1,84 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const statActivityAutovacuumSubsystem = "stat_activity_autovacuum" + +func init() { + registerCollector(statActivityAutovacuumSubsystem, defaultDisabled, NewPGStatActivityAutovacuumCollector) +} + +type PGStatActivityAutovacuumCollector struct { + log log.Logger +} + +func NewPGStatActivityAutovacuumCollector(config collectorConfig) (Collector, error) { + return &PGStatActivityAutovacuumCollector{log: config.logger}, nil +} + +var ( + statActivityAutovacuumAgeInSeconds = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statActivityAutovacuumSubsystem, "timestamp_seconds"), + "Start timestamp of the vacuum process in seconds", + []string{"relname"}, + prometheus.Labels{}, + ) + + statActivityAutovacuumQuery = ` + SELECT + SPLIT_PART(query, '.', 2) AS relname, + EXTRACT(EPOCH FROM xact_start) AS timestamp_seconds + FROM + pg_catalog.pg_stat_activity + WHERE + query LIKE 'autovacuum:%' + ` +) + +func (PGStatActivityAutovacuumCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statActivityAutovacuumQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var relname string + var ageInSeconds float64 + + if err := rows.Scan(&relname, &ageInSeconds); err != nil { + return err + } + + ch <- prometheus.MustNewConstMetric( + statActivityAutovacuumAgeInSeconds, + prometheus.GaugeValue, + ageInSeconds, relname, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_stat_activity_autovacuum_test.go b/collector/pg_stat_activity_autovacuum_test.go new file mode 100644 index 000000000..a6fcdbcad --- /dev/null +++ b/collector/pg_stat_activity_autovacuum_test.go @@ -0,0 +1,62 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatActivityAutovacuumCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "relname", + "timestamp_seconds", + } + rows := sqlmock.NewRows(columns). + AddRow("test", 3600) + + mock.ExpectQuery(sanitizeQuery(statActivityAutovacuumQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatActivityAutovacuumCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatActivityAutovacuumCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"relname": "test"}, value: 3600, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_bgwriter.go b/collector/pg_stat_bgwriter.go index 69c75653f..ec446d58c 100644 --- a/collector/pg_stat_bgwriter.go +++ b/collector/pg_stat_bgwriter.go @@ -16,181 +16,218 @@ package collector import ( "context" "database/sql" - "time" - "github.com/go-kit/log" "github.com/prometheus/client_golang/prometheus" ) +const bgWriterSubsystem = "stat_bgwriter" + func init() { - registerCollector("bgwriter", defaultEnabled, NewPGStatBGWriterCollector) + registerCollector(bgWriterSubsystem, defaultEnabled, NewPGStatBGWriterCollector) } type PGStatBGWriterCollector struct { } -func NewPGStatBGWriterCollector(logger log.Logger) (Collector, error) { +func NewPGStatBGWriterCollector(collectorConfig) (Collector, error) { return &PGStatBGWriterCollector{}, nil } -const bgWriterSubsystem = "stat_bgwriter" - -var statBGWriter = map[string]*prometheus.Desc{ - "checkpoints_timed": prometheus.NewDesc( +var ( + statBGWriterCheckpointsTimedDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "checkpoints_timed_total"), "Number of scheduled checkpoints that have been performed", []string{}, prometheus.Labels{}, - ), - "checkpoints_req": prometheus.NewDesc( + ) + statBGWriterCheckpointsReqDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "checkpoints_req_total"), "Number of requested checkpoints that have been performed", []string{}, prometheus.Labels{}, - ), - "checkpoint_write_time": prometheus.NewDesc( + ) + statBGWriterCheckpointsReqTimeDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "checkpoint_write_time_total"), "Total amount of time that has been spent in the portion of checkpoint processing where files are written to disk, in milliseconds", []string{}, prometheus.Labels{}, - ), - "checkpoint_sync_time": prometheus.NewDesc( + ) + statBGWriterCheckpointsSyncTimeDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "checkpoint_sync_time_total"), "Total amount of time that has been spent in the portion of checkpoint processing where files are synchronized to disk, in milliseconds", []string{}, prometheus.Labels{}, - ), - "buffers_checkpoint": prometheus.NewDesc( + ) + statBGWriterBuffersCheckpointDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "buffers_checkpoint_total"), "Number of buffers written during checkpoints", []string{}, prometheus.Labels{}, - ), - "buffers_clean": prometheus.NewDesc( + ) + statBGWriterBuffersCleanDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "buffers_clean_total"), "Number of buffers written by the background writer", []string{}, prometheus.Labels{}, - ), - "maxwritten_clean": prometheus.NewDesc( + ) + statBGWriterMaxwrittenCleanDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "maxwritten_clean_total"), "Number of times the background writer stopped a cleaning scan because it had written too many buffers", []string{}, prometheus.Labels{}, - ), - "buffers_backend": prometheus.NewDesc( + ) + statBGWriterBuffersBackendDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "buffers_backend_total"), "Number of buffers written directly by a backend", []string{}, prometheus.Labels{}, - ), - "buffers_backend_fsync": prometheus.NewDesc( + ) + statBGWriterBuffersBackendFsyncDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "buffers_backend_fsync_total"), "Number of times a backend had to execute its own fsync call (normally the background writer handles those even when the backend does its own write)", []string{}, prometheus.Labels{}, - ), - "buffers_alloc": prometheus.NewDesc( + ) + statBGWriterBuffersAllocDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "buffers_alloc_total"), "Number of buffers allocated", []string{}, prometheus.Labels{}, - ), - "stats_reset": prometheus.NewDesc( + ) + statBGWriterStatsResetDesc = prometheus.NewDesc( prometheus.BuildFQName(namespace, bgWriterSubsystem, "stats_reset_total"), "Time at which these statistics were last reset", []string{}, prometheus.Labels{}, - ), -} + ) + + statBGWriterQuery = `SELECT + checkpoints_timed + ,checkpoints_req + ,checkpoint_write_time + ,checkpoint_sync_time + ,buffers_checkpoint + ,buffers_clean + ,maxwritten_clean + ,buffers_backend + ,buffers_backend_fsync + ,buffers_alloc + ,stats_reset + FROM pg_stat_bgwriter;` +) -func (PGStatBGWriterCollector) Update(ctx context.Context, db *sql.DB, ch chan<- prometheus.Metric) error { +func (PGStatBGWriterCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() row := db.QueryRowContext(ctx, - `SELECT - checkpoints_timed - ,checkpoints_req - ,checkpoint_write_time - ,checkpoint_sync_time - ,buffers_checkpoint - ,buffers_clean - ,maxwritten_clean - ,buffers_backend - ,buffers_backend_fsync - ,buffers_alloc - ,stats_reset - FROM pg_stat_bgwriter;`) + statBGWriterQuery) - var cpt int - var cpr int - var cpwt float64 - var cpst float64 - var bcp int - var bc int - var mwc int - var bb int - var bbf int - var ba int - var sr time.Time + var cpt, cpr, bcp, bc, mwc, bb, bbf, ba sql.NullInt64 + var cpwt, cpst sql.NullFloat64 + var sr sql.NullTime err := row.Scan(&cpt, &cpr, &cpwt, &cpst, &bcp, &bc, &mwc, &bb, &bbf, &ba, &sr) if err != nil { return err } + cptMetric := 0.0 + if cpt.Valid { + cptMetric = float64(cpt.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["checkpoints_timed"], + statBGWriterCheckpointsTimedDesc, prometheus.CounterValue, - float64(cpt), + cptMetric, ) + cprMetric := 0.0 + if cpr.Valid { + cprMetric = float64(cpr.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["checkpoints_req"], + statBGWriterCheckpointsReqDesc, prometheus.CounterValue, - float64(cpr), + cprMetric, ) + cpwtMetric := 0.0 + if cpwt.Valid { + cpwtMetric = float64(cpwt.Float64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["checkpoint_write_time"], + statBGWriterCheckpointsReqTimeDesc, prometheus.CounterValue, - float64(cpwt), + cpwtMetric, ) + cpstMetric := 0.0 + if cpst.Valid { + cpstMetric = float64(cpst.Float64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["checkpoint_sync_time"], + statBGWriterCheckpointsSyncTimeDesc, prometheus.CounterValue, - float64(cpst), + cpstMetric, ) + bcpMetric := 0.0 + if bcp.Valid { + bcpMetric = float64(bcp.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["buffers_checkpoint"], + statBGWriterBuffersCheckpointDesc, prometheus.CounterValue, - float64(bcp), + bcpMetric, ) + bcMetric := 0.0 + if bc.Valid { + bcMetric = float64(bc.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["buffers_clean"], + statBGWriterBuffersCleanDesc, prometheus.CounterValue, - float64(bc), + bcMetric, ) + mwcMetric := 0.0 + if mwc.Valid { + mwcMetric = float64(mwc.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["maxwritten_clean"], + statBGWriterMaxwrittenCleanDesc, prometheus.CounterValue, - float64(mwc), + mwcMetric, ) + bbMetric := 0.0 + if bb.Valid { + bbMetric = float64(bb.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["buffers_backend"], + statBGWriterBuffersBackendDesc, prometheus.CounterValue, - float64(bb), + bbMetric, ) + bbfMetric := 0.0 + if bbf.Valid { + bbfMetric = float64(bbf.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["buffers_backend_fsync"], + statBGWriterBuffersBackendFsyncDesc, prometheus.CounterValue, - float64(bbf), + bbfMetric, ) + baMetric := 0.0 + if ba.Valid { + baMetric = float64(ba.Int64) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["buffers_alloc"], + statBGWriterBuffersAllocDesc, prometheus.CounterValue, - float64(ba), + baMetric, ) + srMetric := 0.0 + if sr.Valid { + srMetric = float64(sr.Time.Unix()) + } ch <- prometheus.MustNewConstMetric( - statBGWriter["stats_reset"], + statBGWriterStatsResetDesc, prometheus.CounterValue, - float64(sr.Unix()), + srMetric, ) return nil diff --git a/collector/pg_stat_bgwriter_test.go b/collector/pg_stat_bgwriter_test.go new file mode 100644 index 000000000..1c2cf98de --- /dev/null +++ b/collector/pg_stat_bgwriter_test.go @@ -0,0 +1,151 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + "time" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatBGWriterCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "checkpoints_timed", + "checkpoints_req", + "checkpoint_write_time", + "checkpoint_sync_time", + "buffers_checkpoint", + "buffers_clean", + "maxwritten_clean", + "buffers_backend", + "buffers_backend_fsync", + "buffers_alloc", + "stats_reset"} + + srT, err := time.Parse("2006-01-02 15:04:05.00000-07", "2023-05-25 17:10:42.81132-07") + if err != nil { + t.Fatalf("Error parsing time: %s", err) + } + + rows := sqlmock.NewRows(columns). + AddRow(354, 4945, 289097744, 1242257, int64(3275602074), 89320867, 450139, 2034563757, 0, int64(2725688749), srT) + mock.ExpectQuery(sanitizeQuery(statBGWriterQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatBGWriterCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatBGWriterCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 354}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 2725688749}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatBGWriterCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "checkpoints_timed", + "checkpoints_req", + "checkpoint_write_time", + "checkpoint_sync_time", + "buffers_checkpoint", + "buffers_clean", + "maxwritten_clean", + "buffers_backend", + "buffers_backend_fsync", + "buffers_alloc", + "stats_reset"} + + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(statBGWriterQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatBGWriterCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatBGWriterCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_database.go b/collector/pg_stat_database.go new file mode 100644 index 000000000..ea7075303 --- /dev/null +++ b/collector/pg_stat_database.go @@ -0,0 +1,500 @@ +// Copyright 2022 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/go-kit/log/level" + "github.com/prometheus/client_golang/prometheus" +) + +const statDatabaseSubsystem = "stat_database" + +func init() { + registerCollector(statDatabaseSubsystem, defaultEnabled, NewPGStatDatabaseCollector) +} + +type PGStatDatabaseCollector struct { + log log.Logger +} + +func NewPGStatDatabaseCollector(config collectorConfig) (Collector, error) { + return &PGStatDatabaseCollector{log: config.logger}, nil +} + +var ( + statDatabaseNumbackends = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "numbackends", + ), + "Number of backends currently connected to this database. This is the only column in this view that returns a value reflecting current state; all other columns return the accumulated values since the last reset.", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseXactCommit = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "xact_commit", + ), + "Number of transactions in this database that have been committed", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseXactRollback = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "xact_rollback", + ), + "Number of transactions in this database that have been rolled back", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseBlksRead = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "blks_read", + ), + "Number of disk blocks read in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseBlksHit = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "blks_hit", + ), + "Number of times disk blocks were found already in the buffer cache, so that a read was not necessary (this only includes hits in the PostgreSQL buffer cache, not the operating system's file system cache)", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTupReturned = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "tup_returned", + ), + "Number of rows returned by queries in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTupFetched = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "tup_fetched", + ), + "Number of rows fetched by queries in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTupInserted = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "tup_inserted", + ), + "Number of rows inserted by queries in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTupUpdated = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "tup_updated", + ), + "Number of rows updated by queries in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTupDeleted = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "tup_deleted", + ), + "Number of rows deleted by queries in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseConflicts = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "conflicts", + ), + "Number of queries canceled due to conflicts with recovery in this database. (Conflicts occur only on standby servers; see pg_stat_database_conflicts for details.)", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTempFiles = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "temp_files", + ), + "Number of temporary files created by queries in this database. All temporary files are counted, regardless of why the temporary file was created (e.g., sorting or hashing), and regardless of the log_temp_files setting.", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseTempBytes = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "temp_bytes", + ), + "Total amount of data written to temporary files by queries in this database. All temporary files are counted, regardless of why the temporary file was created, and regardless of the log_temp_files setting.", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseDeadlocks = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "deadlocks", + ), + "Number of deadlocks detected in this database", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseBlkReadTime = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "blk_read_time", + ), + "Time spent reading data file blocks by backends in this database, in milliseconds", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseBlkWriteTime = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "blk_write_time", + ), + "Time spent writing data file blocks by backends in this database, in milliseconds", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseStatsReset = prometheus.NewDesc(prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "stats_reset", + ), + "Time at which these statistics were last reset", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + statDatabaseActiveTime = prometheus.NewDesc(prometheus.BuildFQName( + namespace, + statDatabaseSubsystem, + "active_time_seconds_total", + ), + "Time spent executing SQL statements in this database, in seconds", + []string{"datid", "datname"}, + prometheus.Labels{}, + ) + + statDatabaseQuery = ` + SELECT + datid + ,datname + ,numbackends + ,xact_commit + ,xact_rollback + ,blks_read + ,blks_hit + ,tup_returned + ,tup_fetched + ,tup_inserted + ,tup_updated + ,tup_deleted + ,conflicts + ,temp_files + ,temp_bytes + ,deadlocks + ,blk_read_time + ,blk_write_time + ,active_time + ,stats_reset + FROM pg_stat_database; + ` +) + +func (c *PGStatDatabaseCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statDatabaseQuery, + ) + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var datid, datname sql.NullString + var numBackends, xactCommit, xactRollback, blksRead, blksHit, tupReturned, tupFetched, tupInserted, tupUpdated, tupDeleted, conflicts, tempFiles, tempBytes, deadlocks, blkReadTime, blkWriteTime, activeTime sql.NullFloat64 + var statsReset sql.NullTime + + err := rows.Scan( + &datid, + &datname, + &numBackends, + &xactCommit, + &xactRollback, + &blksRead, + &blksHit, + &tupReturned, + &tupFetched, + &tupInserted, + &tupUpdated, + &tupDeleted, + &conflicts, + &tempFiles, + &tempBytes, + &deadlocks, + &blkReadTime, + &blkWriteTime, + &activeTime, + &statsReset, + ) + if err != nil { + return err + } + + if !datid.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no datid") + continue + } + if !datname.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no datname") + continue + } + if !numBackends.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no numbackends") + continue + } + if !xactCommit.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no xact_commit") + continue + } + if !xactRollback.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no xact_rollback") + continue + } + if !blksRead.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no blks_read") + continue + } + if !blksHit.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no blks_hit") + continue + } + if !tupReturned.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no tup_returned") + continue + } + if !tupFetched.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no tup_fetched") + continue + } + if !tupInserted.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no tup_inserted") + continue + } + if !tupUpdated.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no tup_updated") + continue + } + if !tupDeleted.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no tup_deleted") + continue + } + if !conflicts.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no conflicts") + continue + } + if !tempFiles.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no temp_files") + continue + } + if !tempBytes.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no temp_bytes") + continue + } + if !deadlocks.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no deadlocks") + continue + } + if !blkReadTime.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no blk_read_time") + continue + } + if !blkWriteTime.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no blk_write_time") + continue + } + if !activeTime.Valid { + level.Debug(c.log).Log("msg", "Skipping collecting metric because it has no active_time") + continue + } + + statsResetMetric := 0.0 + if !statsReset.Valid { + level.Debug(c.log).Log("msg", "No metric for stats_reset, will collect 0 instead") + } + if statsReset.Valid { + statsResetMetric = float64(statsReset.Time.Unix()) + } + + labels := []string{datid.String, datname.String} + + ch <- prometheus.MustNewConstMetric( + statDatabaseNumbackends, + prometheus.GaugeValue, + numBackends.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseXactCommit, + prometheus.CounterValue, + xactCommit.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseXactRollback, + prometheus.CounterValue, + xactRollback.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseBlksRead, + prometheus.CounterValue, + blksRead.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseBlksHit, + prometheus.CounterValue, + blksHit.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTupReturned, + prometheus.CounterValue, + tupReturned.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTupFetched, + prometheus.CounterValue, + tupFetched.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTupInserted, + prometheus.CounterValue, + tupInserted.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTupUpdated, + prometheus.CounterValue, + tupUpdated.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTupDeleted, + prometheus.CounterValue, + tupDeleted.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseConflicts, + prometheus.CounterValue, + conflicts.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTempFiles, + prometheus.CounterValue, + tempFiles.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseTempBytes, + prometheus.CounterValue, + tempBytes.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseDeadlocks, + prometheus.CounterValue, + deadlocks.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseBlkReadTime, + prometheus.CounterValue, + blkReadTime.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseBlkWriteTime, + prometheus.CounterValue, + blkWriteTime.Float64, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseActiveTime, + prometheus.CounterValue, + activeTime.Float64/1000.0, + labels..., + ) + + ch <- prometheus.MustNewConstMetric( + statDatabaseStatsReset, + prometheus.CounterValue, + statsResetMetric, + labels..., + ) + } + return nil +} diff --git a/collector/pg_stat_database_test.go b/collector/pg_stat_database_test.go new file mode 100644 index 000000000..2d4c25cf4 --- /dev/null +++ b/collector/pg_stat_database_test.go @@ -0,0 +1,523 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + "time" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatDatabaseCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datid", + "datname", + "numbackends", + "xact_commit", + "xact_rollback", + "blks_read", + "blks_hit", + "tup_returned", + "tup_fetched", + "tup_inserted", + "tup_updated", + "tup_deleted", + "conflicts", + "temp_files", + "temp_bytes", + "deadlocks", + "blk_read_time", + "blk_write_time", + "active_time", + "stats_reset", + } + + srT, err := time.Parse("2006-01-02 15:04:05.00000-07", "2023-05-25 17:10:42.81132-07") + if err != nil { + t.Fatalf("Error parsing time: %s", err) + } + + rows := sqlmock.NewRows(columns). + AddRow( + "pid", + "postgres", + 354, + 4945, + 289097744, + 1242257, + int64(3275602074), + 89320867, + 450139, + 2034563757, + 0, + int64(2725688749), + 23, + 52, + 74, + 925, + 16, + 823, + 33, + srT) + + mock.ExpectQuery(sanitizeQuery(statDatabaseQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatDatabaseCollector{ + log: log.With(log.NewNopLogger(), "collector", "pg_stat_database"), + } + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_GAUGE, value: 354}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2725688749}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 23}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 52}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 74}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 925}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 16}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 823}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0.033}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatDatabaseCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + srT, err := time.Parse("2006-01-02 15:04:05.00000-07", "2023-05-25 17:10:42.81132-07") + if err != nil { + t.Fatalf("Error parsing time: %s", err) + } + inst := &instance{db: db} + + columns := []string{ + "datid", + "datname", + "numbackends", + "xact_commit", + "xact_rollback", + "blks_read", + "blks_hit", + "tup_returned", + "tup_fetched", + "tup_inserted", + "tup_updated", + "tup_deleted", + "conflicts", + "temp_files", + "temp_bytes", + "deadlocks", + "blk_read_time", + "blk_write_time", + "active_time", + "stats_reset", + } + + rows := sqlmock.NewRows(columns). + AddRow( + nil, + "postgres", + 354, + 4945, + 289097744, + 1242257, + int64(3275602074), + 89320867, + 450139, + 2034563757, + 0, + int64(2725688749), + 23, + 52, + 74, + 925, + 16, + 823, + 32, + srT). + AddRow( + "pid", + "postgres", + 354, + 4945, + 289097744, + 1242257, + int64(3275602074), + 89320867, + 450139, + 2034563757, + 0, + int64(2725688749), + 23, + 52, + 74, + 925, + 16, + 823, + 32, + srT) + mock.ExpectQuery(sanitizeQuery(statDatabaseQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatDatabaseCollector{ + log: log.With(log.NewNopLogger(), "collector", "pg_stat_database"), + } + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_GAUGE, value: 354}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2725688749}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 23}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 52}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 74}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 925}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 16}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 823}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0.032}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} +func TestPGStatDatabaseCollectorRowLeakTest(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datid", + "datname", + "numbackends", + "xact_commit", + "xact_rollback", + "blks_read", + "blks_hit", + "tup_returned", + "tup_fetched", + "tup_inserted", + "tup_updated", + "tup_deleted", + "conflicts", + "temp_files", + "temp_bytes", + "deadlocks", + "blk_read_time", + "blk_write_time", + "active_time", + "stats_reset", + } + + srT, err := time.Parse("2006-01-02 15:04:05.00000-07", "2023-05-25 17:10:42.81132-07") + if err != nil { + t.Fatalf("Error parsing time: %s", err) + } + + rows := sqlmock.NewRows(columns). + AddRow( + "pid", + "postgres", + 354, + 4945, + 289097744, + 1242257, + int64(3275602074), + 89320867, + 450139, + 2034563757, + 0, + int64(2725688749), + 23, + 52, + 74, + 925, + 16, + 823, + 14, + srT). + AddRow( + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + ). + AddRow( + "pid", + "postgres", + 355, + 4946, + 289097745, + 1242258, + int64(3275602075), + 89320868, + 450140, + 2034563758, + 1, + int64(2725688750), + 24, + 53, + 75, + 926, + 17, + 824, + 15, + srT) + mock.ExpectQuery(sanitizeQuery(statDatabaseQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatDatabaseCollector{ + log: log.With(log.NewNopLogger(), "collector", "pg_stat_database"), + } + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_GAUGE, value: 354}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2725688749}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 23}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 52}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 74}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 925}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 16}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 823}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0.014}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_GAUGE, value: 355}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 4946}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 289097745}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1242258}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 3275602075}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 89320868}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 450140}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2034563758}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2725688750}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 24}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 53}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 75}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 926}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 17}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 824}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0.015}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1685059842}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatDatabaseCollectorTestNilStatReset(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datid", + "datname", + "numbackends", + "xact_commit", + "xact_rollback", + "blks_read", + "blks_hit", + "tup_returned", + "tup_fetched", + "tup_inserted", + "tup_updated", + "tup_deleted", + "conflicts", + "temp_files", + "temp_bytes", + "deadlocks", + "blk_read_time", + "blk_write_time", + "active_time", + "stats_reset", + } + + rows := sqlmock.NewRows(columns). + AddRow( + "pid", + "postgres", + 354, + 4945, + 289097744, + 1242257, + int64(3275602074), + 89320867, + 450139, + 2034563757, + 0, + int64(2725688749), + 23, + 52, + 74, + 925, + 16, + 823, + 7, + nil) + + mock.ExpectQuery(sanitizeQuery(statDatabaseQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatDatabaseCollector{ + log: log.With(log.NewNopLogger(), "collector", "pg_stat_database"), + } + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatDatabaseCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_GAUGE, value: 354}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 4945}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 289097744}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 1242257}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 3275602074}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 89320867}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 450139}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2034563757}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 2725688749}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 23}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 52}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 74}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 925}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 16}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 823}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0.007}, + {labels: labelMap{"datid": "pid", "datname": "postgres"}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_statements.go b/collector/pg_stat_statements.go new file mode 100644 index 000000000..c03e78b92 --- /dev/null +++ b/collector/pg_stat_statements.go @@ -0,0 +1,211 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/blang/semver/v4" + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const statStatementsSubsystem = "stat_statements" + +func init() { + // WARNING: + // Disabled by default because this set of metrics can be quite expensive on a busy server + // Every unique query will cause a new timeseries to be created + registerCollector(statStatementsSubsystem, defaultDisabled, NewPGStatStatementsCollector) +} + +type PGStatStatementsCollector struct { + log log.Logger +} + +func NewPGStatStatementsCollector(config collectorConfig) (Collector, error) { + return &PGStatStatementsCollector{log: config.logger}, nil +} + +var ( + statSTatementsCallsTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "calls_total"), + "Number of times executed", + []string{"user", "datname", "queryid"}, + prometheus.Labels{}, + ) + statStatementsSecondsTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "seconds_total"), + "Total time spent in the statement, in seconds", + []string{"user", "datname", "queryid"}, + prometheus.Labels{}, + ) + statStatementsRowsTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "rows_total"), + "Total number of rows retrieved or affected by the statement", + []string{"user", "datname", "queryid"}, + prometheus.Labels{}, + ) + statStatementsBlockReadSecondsTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "block_read_seconds_total"), + "Total time the statement spent reading blocks, in seconds", + []string{"user", "datname", "queryid"}, + prometheus.Labels{}, + ) + statStatementsBlockWriteSecondsTotal = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statStatementsSubsystem, "block_write_seconds_total"), + "Total time the statement spent writing blocks, in seconds", + []string{"user", "datname", "queryid"}, + prometheus.Labels{}, + ) + + pgStatStatementsQuery = `SELECT + pg_get_userbyid(userid) as user, + pg_database.datname, + pg_stat_statements.queryid, + pg_stat_statements.calls as calls_total, + pg_stat_statements.total_time / 1000.0 as seconds_total, + pg_stat_statements.rows as rows_total, + pg_stat_statements.blk_read_time / 1000.0 as block_read_seconds_total, + pg_stat_statements.blk_write_time / 1000.0 as block_write_seconds_total + FROM pg_stat_statements + JOIN pg_database + ON pg_database.oid = pg_stat_statements.dbid + WHERE + total_time > ( + SELECT percentile_cont(0.1) + WITHIN GROUP (ORDER BY total_time) + FROM pg_stat_statements + ) + ORDER BY seconds_total DESC + LIMIT 100;` + + pgStatStatementsNewQuery = `SELECT + pg_get_userbyid(userid) as user, + pg_database.datname, + pg_stat_statements.queryid, + pg_stat_statements.calls as calls_total, + pg_stat_statements.total_exec_time / 1000.0 as seconds_total, + pg_stat_statements.rows as rows_total, + pg_stat_statements.blk_read_time / 1000.0 as block_read_seconds_total, + pg_stat_statements.blk_write_time / 1000.0 as block_write_seconds_total + FROM pg_stat_statements + JOIN pg_database + ON pg_database.oid = pg_stat_statements.dbid + WHERE + total_exec_time > ( + SELECT percentile_cont(0.1) + WITHIN GROUP (ORDER BY total_exec_time) + FROM pg_stat_statements + ) + ORDER BY seconds_total DESC + LIMIT 100;` +) + +func (PGStatStatementsCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + query := pgStatStatementsQuery + if instance.version.GE(semver.MustParse("13.0.0")) { + query = pgStatStatementsNewQuery + } + + db := instance.getDB() + rows, err := db.QueryContext(ctx, query) + + if err != nil { + return err + } + defer rows.Close() + for rows.Next() { + var user, datname, queryid sql.NullString + var callsTotal, rowsTotal sql.NullInt64 + var secondsTotal, blockReadSecondsTotal, blockWriteSecondsTotal sql.NullFloat64 + + if err := rows.Scan(&user, &datname, &queryid, &callsTotal, &secondsTotal, &rowsTotal, &blockReadSecondsTotal, &blockWriteSecondsTotal); err != nil { + return err + } + + userLabel := "unknown" + if user.Valid { + userLabel = user.String + } + datnameLabel := "unknown" + if datname.Valid { + datnameLabel = datname.String + } + queryidLabel := "unknown" + if queryid.Valid { + queryidLabel = queryid.String + } + + callsTotalMetric := 0.0 + if callsTotal.Valid { + callsTotalMetric = float64(callsTotal.Int64) + } + ch <- prometheus.MustNewConstMetric( + statSTatementsCallsTotal, + prometheus.CounterValue, + callsTotalMetric, + userLabel, datnameLabel, queryidLabel, + ) + + secondsTotalMetric := 0.0 + if secondsTotal.Valid { + secondsTotalMetric = secondsTotal.Float64 + } + ch <- prometheus.MustNewConstMetric( + statStatementsSecondsTotal, + prometheus.CounterValue, + secondsTotalMetric, + userLabel, datnameLabel, queryidLabel, + ) + + rowsTotalMetric := 0.0 + if rowsTotal.Valid { + rowsTotalMetric = float64(rowsTotal.Int64) + } + ch <- prometheus.MustNewConstMetric( + statStatementsRowsTotal, + prometheus.CounterValue, + rowsTotalMetric, + userLabel, datnameLabel, queryidLabel, + ) + + blockReadSecondsTotalMetric := 0.0 + if blockReadSecondsTotal.Valid { + blockReadSecondsTotalMetric = blockReadSecondsTotal.Float64 + } + ch <- prometheus.MustNewConstMetric( + statStatementsBlockReadSecondsTotal, + prometheus.CounterValue, + blockReadSecondsTotalMetric, + userLabel, datnameLabel, queryidLabel, + ) + + blockWriteSecondsTotalMetric := 0.0 + if blockWriteSecondsTotal.Valid { + blockWriteSecondsTotalMetric = blockWriteSecondsTotal.Float64 + } + ch <- prometheus.MustNewConstMetric( + statStatementsBlockWriteSecondsTotal, + prometheus.CounterValue, + blockWriteSecondsTotalMetric, + userLabel, datnameLabel, queryidLabel, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_stat_statements_test.go b/collector/pg_stat_statements_test.go new file mode 100644 index 000000000..08aba34c2 --- /dev/null +++ b/collector/pg_stat_statements_test.go @@ -0,0 +1,153 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/blang/semver/v4" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStateStatementsCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("12.0.0")} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(pgStatStatementsQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStateStatementsCollectorNull(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil, nil, nil, nil) + mock.ExpectQuery(sanitizeQuery(pgStatStatementsNewQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"user": "unknown", "datname": "unknown", "queryid": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStateStatementsCollectorNewPG(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db, version: semver.MustParse("13.3.7")} + + columns := []string{"user", "datname", "queryid", "calls_total", "seconds_total", "rows_total", "block_read_seconds_total", "block_write_seconds_total"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", "postgres", 1500, 5, 0.4, 100, 0.1, 0.2) + mock.ExpectQuery(sanitizeQuery(pgStatStatementsNewQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatStatementsCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatStatementsCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.4}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 100}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.1}, + {labels: labelMap{"user": "postgres", "datname": "postgres", "queryid": "1500"}, metricType: dto.MetricType_COUNTER, value: 0.2}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_user_tables.go b/collector/pg_stat_user_tables.go new file mode 100644 index 000000000..af3822ca8 --- /dev/null +++ b/collector/pg_stat_user_tables.go @@ -0,0 +1,446 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const userTableSubsystem = "stat_user_tables" + +func init() { + registerCollector(userTableSubsystem, defaultEnabled, NewPGStatUserTablesCollector) +} + +type PGStatUserTablesCollector struct { + log log.Logger +} + +func NewPGStatUserTablesCollector(config collectorConfig) (Collector, error) { + return &PGStatUserTablesCollector{log: config.logger}, nil +} + +var ( + statUserTablesSeqScan = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "seq_scan"), + "Number of sequential scans initiated on this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesSeqTupRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "seq_tup_read"), + "Number of live rows fetched by sequential scans", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesIdxScan = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "idx_scan"), + "Number of index scans initiated on this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesIdxTupFetch = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "idx_tup_fetch"), + "Number of live rows fetched by index scans", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNTupIns = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_tup_ins"), + "Number of rows inserted", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNTupUpd = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_tup_upd"), + "Number of rows updated", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNTupDel = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_tup_del"), + "Number of rows deleted", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNTupHotUpd = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_tup_hot_upd"), + "Number of rows HOT updated (i.e., with no separate index update required)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNLiveTup = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_live_tup"), + "Estimated number of live rows", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNDeadTup = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_dead_tup"), + "Estimated number of dead rows", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesNModSinceAnalyze = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "n_mod_since_analyze"), + "Estimated number of rows changed since last analyze", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesLastVacuum = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "last_vacuum"), + "Last time at which this table was manually vacuumed (not counting VACUUM FULL)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesLastAutovacuum = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "last_autovacuum"), + "Last time at which this table was vacuumed by the autovacuum daemon", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesLastAnalyze = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "last_analyze"), + "Last time at which this table was manually analyzed", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesLastAutoanalyze = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "last_autoanalyze"), + "Last time at which this table was analyzed by the autovacuum daemon", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesVacuumCount = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "vacuum_count"), + "Number of times this table has been manually vacuumed (not counting VACUUM FULL)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesAutovacuumCount = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "autovacuum_count"), + "Number of times this table has been vacuumed by the autovacuum daemon", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesAnalyzeCount = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "analyze_count"), + "Number of times this table has been manually analyzed", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesAutoanalyzeCount = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "autoanalyze_count"), + "Number of times this table has been analyzed by the autovacuum daemon", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statUserTablesTotalSize = prometheus.NewDesc( + prometheus.BuildFQName(namespace, userTableSubsystem, "size_bytes"), + "Total disk space used by this table, in bytes, including all indexes and TOAST data", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + + statUserTablesQuery = `SELECT + current_database() datname, + schemaname, + relname, + seq_scan, + seq_tup_read, + idx_scan, + idx_tup_fetch, + n_tup_ins, + n_tup_upd, + n_tup_del, + n_tup_hot_upd, + n_live_tup, + n_dead_tup, + n_mod_since_analyze, + COALESCE(last_vacuum, '1970-01-01Z') as last_vacuum, + COALESCE(last_autovacuum, '1970-01-01Z') as last_autovacuum, + COALESCE(last_analyze, '1970-01-01Z') as last_analyze, + COALESCE(last_autoanalyze, '1970-01-01Z') as last_autoanalyze, + vacuum_count, + autovacuum_count, + analyze_count, + autoanalyze_count, + pg_total_relation_size(relid) as total_size + FROM + pg_stat_user_tables` +) + +func (c *PGStatUserTablesCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statUserTablesQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var datname, schemaname, relname sql.NullString + var seqScan, seqTupRead, idxScan, idxTupFetch, nTupIns, nTupUpd, nTupDel, nTupHotUpd, nLiveTup, nDeadTup, + nModSinceAnalyze, vacuumCount, autovacuumCount, analyzeCount, autoanalyzeCount, totalSize sql.NullInt64 + var lastVacuum, lastAutovacuum, lastAnalyze, lastAutoanalyze sql.NullTime + + if err := rows.Scan(&datname, &schemaname, &relname, &seqScan, &seqTupRead, &idxScan, &idxTupFetch, &nTupIns, &nTupUpd, &nTupDel, &nTupHotUpd, &nLiveTup, &nDeadTup, &nModSinceAnalyze, &lastVacuum, &lastAutovacuum, &lastAnalyze, &lastAutoanalyze, &vacuumCount, &autovacuumCount, &analyzeCount, &autoanalyzeCount, &totalSize); err != nil { + return err + } + + datnameLabel := "unknown" + if datname.Valid { + datnameLabel = datname.String + } + schemanameLabel := "unknown" + if schemaname.Valid { + schemanameLabel = schemaname.String + } + relnameLabel := "unknown" + if relname.Valid { + relnameLabel = relname.String + } + + seqScanMetric := 0.0 + if seqScan.Valid { + seqScanMetric = float64(seqScan.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesSeqScan, + prometheus.CounterValue, + seqScanMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + seqTupReadMetric := 0.0 + if seqTupRead.Valid { + seqTupReadMetric = float64(seqTupRead.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesSeqTupRead, + prometheus.CounterValue, + seqTupReadMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + idxScanMetric := 0.0 + if idxScan.Valid { + idxScanMetric = float64(idxScan.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesIdxScan, + prometheus.CounterValue, + idxScanMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + idxTupFetchMetric := 0.0 + if idxTupFetch.Valid { + idxTupFetchMetric = float64(idxTupFetch.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesIdxTupFetch, + prometheus.CounterValue, + idxTupFetchMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nTupInsMetric := 0.0 + if nTupIns.Valid { + nTupInsMetric = float64(nTupIns.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNTupIns, + prometheus.CounterValue, + nTupInsMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nTupUpdMetric := 0.0 + if nTupUpd.Valid { + nTupUpdMetric = float64(nTupUpd.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNTupUpd, + prometheus.CounterValue, + nTupUpdMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nTupDelMetric := 0.0 + if nTupDel.Valid { + nTupDelMetric = float64(nTupDel.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNTupDel, + prometheus.CounterValue, + nTupDelMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nTupHotUpdMetric := 0.0 + if nTupHotUpd.Valid { + nTupHotUpdMetric = float64(nTupHotUpd.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNTupHotUpd, + prometheus.CounterValue, + nTupHotUpdMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nLiveTupMetric := 0.0 + if nLiveTup.Valid { + nLiveTupMetric = float64(nLiveTup.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNLiveTup, + prometheus.GaugeValue, + nLiveTupMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nDeadTupMetric := 0.0 + if nDeadTup.Valid { + nDeadTupMetric = float64(nDeadTup.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNDeadTup, + prometheus.GaugeValue, + nDeadTupMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + nModSinceAnalyzeMetric := 0.0 + if nModSinceAnalyze.Valid { + nModSinceAnalyzeMetric = float64(nModSinceAnalyze.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesNModSinceAnalyze, + prometheus.GaugeValue, + nModSinceAnalyzeMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + lastVacuumMetric := 0.0 + if lastVacuum.Valid { + lastVacuumMetric = float64(lastVacuum.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesLastVacuum, + prometheus.GaugeValue, + lastVacuumMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + lastAutovacuumMetric := 0.0 + if lastAutovacuum.Valid { + lastAutovacuumMetric = float64(lastAutovacuum.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesLastAutovacuum, + prometheus.GaugeValue, + lastAutovacuumMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + lastAnalyzeMetric := 0.0 + if lastAnalyze.Valid { + lastAnalyzeMetric = float64(lastAnalyze.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesLastAnalyze, + prometheus.GaugeValue, + lastAnalyzeMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + lastAutoanalyzeMetric := 0.0 + if lastAutoanalyze.Valid { + lastAutoanalyzeMetric = float64(lastAutoanalyze.Time.Unix()) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesLastAutoanalyze, + prometheus.GaugeValue, + lastAutoanalyzeMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + vacuumCountMetric := 0.0 + if vacuumCount.Valid { + vacuumCountMetric = float64(vacuumCount.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesVacuumCount, + prometheus.CounterValue, + vacuumCountMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + autovacuumCountMetric := 0.0 + if autovacuumCount.Valid { + autovacuumCountMetric = float64(autovacuumCount.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesAutovacuumCount, + prometheus.CounterValue, + autovacuumCountMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + analyzeCountMetric := 0.0 + if analyzeCount.Valid { + analyzeCountMetric = float64(analyzeCount.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesAnalyzeCount, + prometheus.CounterValue, + analyzeCountMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + autoanalyzeCountMetric := 0.0 + if autoanalyzeCount.Valid { + autoanalyzeCountMetric = float64(autoanalyzeCount.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesAutoanalyzeCount, + prometheus.CounterValue, + autoanalyzeCountMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + totalSizeMetric := 0.0 + if totalSize.Valid { + totalSizeMetric = float64(totalSize.Int64) + } + ch <- prometheus.MustNewConstMetric( + statUserTablesTotalSize, + prometheus.GaugeValue, + totalSizeMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + } + + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_stat_user_tables_test.go b/collector/pg_stat_user_tables_test.go new file mode 100644 index 000000000..5e82335c3 --- /dev/null +++ b/collector/pg_stat_user_tables_test.go @@ -0,0 +1,243 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + "time" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatUserTablesCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + lastVacuumTime, err := time.Parse("2006-01-02Z", "2023-06-02Z") + if err != nil { + t.Fatalf("Error parsing vacuum time: %s", err) + } + lastAutoVacuumTime, err := time.Parse("2006-01-02Z", "2023-06-03Z") + if err != nil { + t.Fatalf("Error parsing vacuum time: %s", err) + } + lastAnalyzeTime, err := time.Parse("2006-01-02Z", "2023-06-04Z") + if err != nil { + t.Fatalf("Error parsing vacuum time: %s", err) + } + lastAutoAnalyzeTime, err := time.Parse("2006-01-02Z", "2023-06-05Z") + if err != nil { + t.Fatalf("Error parsing vacuum time: %s", err) + } + + columns := []string{ + "datname", + "schemaname", + "relname", + "seq_scan", + "seq_tup_read", + "idx_scan", + "idx_tup_fetch", + "n_tup_ins", + "n_tup_upd", + "n_tup_del", + "n_tup_hot_upd", + "n_live_tup", + "n_dead_tup", + "n_mod_since_analyze", + "last_vacuum", + "last_autovacuum", + "last_analyze", + "last_autoanalyze", + "vacuum_count", + "autovacuum_count", + "analyze_count", + "autoanalyze_count", + "total_size"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", + "public", + "a_table", + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 0, + lastVacuumTime, + lastAutoVacuumTime, + lastAnalyzeTime, + lastAutoAnalyzeTime, + 11, + 12, + 13, + 14, + 15) + mock.ExpectQuery(sanitizeQuery(statUserTablesQuery)).WillReturnRows(rows) + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatUserTablesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatUserTablesCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 1}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 2}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 3}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 4}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 6}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 7}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 8}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 9}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 10}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 1685664000}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 1685750400}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 1685836800}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_GAUGE, value: 1685923200}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 11}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 12}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 13}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 14}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatUserTablesCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datname", + "schemaname", + "relname", + "seq_scan", + "seq_tup_read", + "idx_scan", + "idx_tup_fetch", + "n_tup_ins", + "n_tup_upd", + "n_tup_del", + "n_tup_hot_upd", + "n_live_tup", + "n_dead_tup", + "n_mod_since_analyze", + "last_vacuum", + "last_autovacuum", + "last_analyze", + "last_autoanalyze", + "vacuum_count", + "autovacuum_count", + "analyze_count", + "autoanalyze_count", + "total_size"} + rows := sqlmock.NewRows(columns). + AddRow("postgres", + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil) + mock.ExpectQuery(sanitizeQuery(statUserTablesQuery)).WillReturnRows(rows) + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatUserTablesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatUserTablesCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_GAUGE, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "postgres", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_stat_walreceiver.go b/collector/pg_stat_walreceiver.go new file mode 100644 index 000000000..db533ab55 --- /dev/null +++ b/collector/pg_stat_walreceiver.go @@ -0,0 +1,271 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "database/sql" + "fmt" + + "github.com/go-kit/log" + "github.com/go-kit/log/level" + "github.com/prometheus/client_golang/prometheus" +) + +func init() { + registerCollector(statWalReceiverSubsystem, defaultDisabled, NewPGStatWalReceiverCollector) +} + +type PGStatWalReceiverCollector struct { + log log.Logger +} + +const statWalReceiverSubsystem = "stat_wal_receiver" + +func NewPGStatWalReceiverCollector(config collectorConfig) (Collector, error) { + return &PGStatWalReceiverCollector{log: config.logger}, nil +} + +var ( + labelCats = []string{"upstream_host", "slot_name", "status"} + statWalReceiverReceiveStartLsn = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "receive_start_lsn"), + "First write-ahead log location used when WAL receiver is started represented as a decimal", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverReceiveStartTli = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "receive_start_tli"), + "First timeline number used when WAL receiver is started", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverFlushedLSN = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "flushed_lsn"), + "Last write-ahead log location already received and flushed to disk, the initial value of this field being the first log location used when WAL receiver is started represented as a decimal", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverReceivedTli = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "received_tli"), + "Timeline number of last write-ahead log location received and flushed to disk", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverLastMsgSendTime = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "last_msg_send_time"), + "Send time of last message received from origin WAL sender", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverLastMsgReceiptTime = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "last_msg_receipt_time"), + "Send time of last message received from origin WAL sender", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverLatestEndLsn = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "latest_end_lsn"), + "Last write-ahead log location reported to origin WAL sender as integer", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverLatestEndTime = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "latest_end_time"), + "Time of last write-ahead log location reported to origin WAL sender", + labelCats, + prometheus.Labels{}, + ) + statWalReceiverUpstreamNode = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statWalReceiverSubsystem, "upstream_node"), + "Node ID of the upstream node", + labelCats, + prometheus.Labels{}, + ) + + pgStatWalColumnQuery = ` + SELECT + column_name + FROM information_schema.columns + WHERE + table_name = 'pg_stat_wal_receiver' and + column_name = 'flushed_lsn' + ` + + pgStatWalReceiverQueryTemplate = ` + SELECT + trim(both '''' from substring(conninfo from 'host=([^ ]*)')) as upstream_host, + slot_name, + status, + (receive_start_lsn- '0/0') %% (2^52)::bigint as receive_start_lsn, + %s +receive_start_tli, + received_tli, + extract(epoch from last_msg_send_time) as last_msg_send_time, + extract(epoch from last_msg_receipt_time) as last_msg_receipt_time, + (latest_end_lsn - '0/0') %% (2^52)::bigint as latest_end_lsn, + extract(epoch from latest_end_time) as latest_end_time, + substring(slot_name from 'repmgr_slot_([0-9]*)') as upstream_node + FROM pg_catalog.pg_stat_wal_receiver + ` +) + +func (c *PGStatWalReceiverCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + hasFlushedLSNRows, err := db.QueryContext(ctx, pgStatWalColumnQuery) + if err != nil { + return err + } + + hasFlushedLSN := hasFlushedLSNRows.Next() + var query string + if hasFlushedLSN { + query = fmt.Sprintf(pgStatWalReceiverQueryTemplate, "(flushed_lsn - '0/0') % (2^52)::bigint as flushed_lsn,\n") + } else { + query = fmt.Sprintf(pgStatWalReceiverQueryTemplate, "") + } + + hasFlushedLSNRows.Close() + + rows, err := db.QueryContext(ctx, query) + if err != nil { + return err + } + defer rows.Close() + for rows.Next() { + var upstreamHost, slotName, status sql.NullString + var receiveStartLsn, receiveStartTli, flushedLsn, receivedTli, latestEndLsn, upstreamNode sql.NullInt64 + var lastMsgSendTime, lastMsgReceiptTime, latestEndTime sql.NullFloat64 + + if hasFlushedLSN { + if err := rows.Scan(&upstreamHost, &slotName, &status, &receiveStartLsn, &receiveStartTli, &flushedLsn, &receivedTli, &lastMsgSendTime, &lastMsgReceiptTime, &latestEndLsn, &latestEndTime, &upstreamNode); err != nil { + return err + } + } else { + if err := rows.Scan(&upstreamHost, &slotName, &status, &receiveStartLsn, &receiveStartTli, &receivedTli, &lastMsgSendTime, &lastMsgReceiptTime, &latestEndLsn, &latestEndTime, &upstreamNode); err != nil { + return err + } + } + if !upstreamHost.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because upstream host is null") + continue + } + + if !slotName.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because slotname host is null") + continue + } + + if !status.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because status is null") + continue + } + labels := []string{upstreamHost.String, slotName.String, status.String} + + if !receiveStartLsn.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because receive_start_lsn is null") + continue + } + if !receiveStartTli.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because receive_start_tli is null") + continue + } + if hasFlushedLSN && !flushedLsn.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because flushed_lsn is null") + continue + } + if !receivedTli.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because received_tli is null") + continue + } + if !lastMsgSendTime.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because last_msg_send_time is null") + continue + } + if !lastMsgReceiptTime.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because last_msg_receipt_time is null") + continue + } + if !latestEndLsn.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because latest_end_lsn is null") + continue + } + if !latestEndTime.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because latest_end_time is null") + continue + } + if !upstreamNode.Valid { + level.Debug(c.log).Log("msg", "Skipping wal receiver stats because upstream_node is null") + continue + } + ch <- prometheus.MustNewConstMetric( + statWalReceiverReceiveStartLsn, + prometheus.CounterValue, + float64(receiveStartLsn.Int64), + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverReceiveStartTli, + prometheus.GaugeValue, + float64(receiveStartTli.Int64), + labels...) + + if hasFlushedLSN { + ch <- prometheus.MustNewConstMetric( + statWalReceiverFlushedLSN, + prometheus.CounterValue, + float64(flushedLsn.Int64), + labels...) + } + + ch <- prometheus.MustNewConstMetric( + statWalReceiverReceivedTli, + prometheus.GaugeValue, + float64(receivedTli.Int64), + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverLastMsgSendTime, + prometheus.CounterValue, + float64(lastMsgSendTime.Float64), + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverLastMsgReceiptTime, + prometheus.CounterValue, + float64(lastMsgReceiptTime.Float64), + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverLatestEndLsn, + prometheus.CounterValue, + float64(latestEndLsn.Int64), + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverLatestEndTime, + prometheus.CounterValue, + latestEndTime.Float64, + labels...) + + ch <- prometheus.MustNewConstMetric( + statWalReceiverUpstreamNode, + prometheus.GaugeValue, + float64(upstreamNode.Int64), + labels...) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_stat_walreceiver_test.go b/collector/pg_stat_walreceiver_test.go new file mode 100644 index 000000000..c81c9ecae --- /dev/null +++ b/collector/pg_stat_walreceiver_test.go @@ -0,0 +1,186 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "fmt" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +var queryWithFlushedLSN = fmt.Sprintf(pgStatWalReceiverQueryTemplate, "(flushed_lsn - '0/0') % (2^52)::bigint as flushed_lsn,\n") +var queryWithNoFlushedLSN = fmt.Sprintf(pgStatWalReceiverQueryTemplate, "") + +func TestPGStatWalReceiverCollectorWithFlushedLSN(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + infoSchemaColumns := []string{ + "column_name", + } + + infoSchemaRows := sqlmock.NewRows(infoSchemaColumns). + AddRow( + "flushed_lsn", + ) + + mock.ExpectQuery(sanitizeQuery(pgStatWalColumnQuery)).WillReturnRows(infoSchemaRows) + + columns := []string{ + "upstream_host", + "slot_name", + "status", + "receive_start_lsn", + "receive_start_tli", + "flushed_lsn", + "received_tli", + "last_msg_send_time", + "last_msg_receipt_time", + "latest_end_lsn", + "latest_end_time", + "upstream_node", + } + rows := sqlmock.NewRows(columns). + AddRow( + "foo", + "bar", + "stopping", + int64(1200668684563608), + 1687321285, + int64(1200668684563609), + 1687321280, + 1687321275, + 1687321276, + int64(1200668684563610), + 1687321277, + 5, + ) + + mock.ExpectQuery(sanitizeQuery(queryWithFlushedLSN)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatWalReceiverCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PgStatWalReceiverCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1200668684563608, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321285, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1200668684563609, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321280, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321275, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321276, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1200668684563610, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 1687321277, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "stopping"}, value: 5, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } + +} + +func TestPGStatWalReceiverCollectorWithNoFlushedLSN(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + infoSchemaColumns := []string{ + "column_name", + } + + infoSchemaRows := sqlmock.NewRows(infoSchemaColumns) + + mock.ExpectQuery(sanitizeQuery(pgStatWalColumnQuery)).WillReturnRows(infoSchemaRows) + + columns := []string{ + "upstream_host", + "slot_name", + "status", + "receive_start_lsn", + "receive_start_tli", + "received_tli", + "last_msg_send_time", + "last_msg_receipt_time", + "latest_end_lsn", + "latest_end_time", + "upstream_node", + } + rows := sqlmock.NewRows(columns). + AddRow( + "foo", + "bar", + "starting", + int64(1200668684563608), + 1687321285, + 1687321280, + 1687321275, + 1687321276, + int64(1200668684563610), + 1687321277, + 5, + ) + mock.ExpectQuery(sanitizeQuery(queryWithNoFlushedLSN)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatWalReceiverCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PgStatWalReceiverCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1200668684563608, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1687321285, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1687321280, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1687321275, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1687321276, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1200668684563610, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 1687321277, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"upstream_host": "foo", "slot_name": "bar", "status": "starting"}, value: 5, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } + +} diff --git a/collector/pg_statio_user_indexes.go b/collector/pg_statio_user_indexes.go new file mode 100644 index 000000000..b5516338d --- /dev/null +++ b/collector/pg_statio_user_indexes.go @@ -0,0 +1,118 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +func init() { + registerCollector(statioUserIndexesSubsystem, defaultDisabled, NewPGStatioUserIndexesCollector) +} + +type PGStatioUserIndexesCollector struct { + log log.Logger +} + +const statioUserIndexesSubsystem = "statio_user_indexes" + +func NewPGStatioUserIndexesCollector(config collectorConfig) (Collector, error) { + return &PGStatioUserIndexesCollector{log: config.logger}, nil +} + +var ( + statioUserIndexesIdxBlksRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserIndexesSubsystem, "idx_blks_read_total"), + "Number of disk blocks read from this index", + []string{"schemaname", "relname", "indexrelname"}, + prometheus.Labels{}, + ) + statioUserIndexesIdxBlksHit = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserIndexesSubsystem, "idx_blks_hit_total"), + "Number of buffer hits in this index", + []string{"schemaname", "relname", "indexrelname"}, + prometheus.Labels{}, + ) + + statioUserIndexesQuery = ` + SELECT + schemaname, + relname, + indexrelname, + idx_blks_read, + idx_blks_hit + FROM pg_statio_user_indexes + ` +) + +func (c *PGStatioUserIndexesCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statioUserIndexesQuery) + + if err != nil { + return err + } + defer rows.Close() + for rows.Next() { + var schemaname, relname, indexrelname sql.NullString + var idxBlksRead, idxBlksHit sql.NullFloat64 + + if err := rows.Scan(&schemaname, &relname, &indexrelname, &idxBlksRead, &idxBlksHit); err != nil { + return err + } + schemanameLabel := "unknown" + if schemaname.Valid { + schemanameLabel = schemaname.String + } + relnameLabel := "unknown" + if relname.Valid { + relnameLabel = relname.String + } + indexrelnameLabel := "unknown" + if indexrelname.Valid { + indexrelnameLabel = indexrelname.String + } + labels := []string{schemanameLabel, relnameLabel, indexrelnameLabel} + + idxBlksReadMetric := 0.0 + if idxBlksRead.Valid { + idxBlksReadMetric = idxBlksRead.Float64 + } + ch <- prometheus.MustNewConstMetric( + statioUserIndexesIdxBlksRead, + prometheus.CounterValue, + idxBlksReadMetric, + labels..., + ) + + idxBlksHitMetric := 0.0 + if idxBlksHit.Valid { + idxBlksHitMetric = idxBlksHit.Float64 + } + ch <- prometheus.MustNewConstMetric( + statioUserIndexesIdxBlksHit, + prometheus.CounterValue, + idxBlksHitMetric, + labels..., + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_statio_user_indexes_test.go b/collector/pg_statio_user_indexes_test.go new file mode 100644 index 000000000..174012162 --- /dev/null +++ b/collector/pg_statio_user_indexes_test.go @@ -0,0 +1,109 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPgStatioUserIndexesCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "schemaname", + "relname", + "indexrelname", + "idx_blks_read", + "idx_blks_hit", + } + rows := sqlmock.NewRows(columns). + AddRow("public", "pgtest_accounts", "pgtest_accounts_pkey", 8, 9) + + mock.ExpectQuery(sanitizeQuery(statioUserIndexesQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatioUserIndexesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatioUserIndexesCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"schemaname": "public", "relname": "pgtest_accounts", "indexrelname": "pgtest_accounts_pkey"}, value: 8, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"schemaname": "public", "relname": "pgtest_accounts", "indexrelname": "pgtest_accounts_pkey"}, value: 9, metricType: dto.MetricType_COUNTER}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPgStatioUserIndexesCollectorNull(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "schemaname", + "relname", + "indexrelname", + "idx_blks_read", + "idx_blks_hit", + } + rows := sqlmock.NewRows(columns). + AddRow(nil, nil, nil, nil, nil) + + mock.ExpectQuery(sanitizeQuery(statioUserIndexesQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatioUserIndexesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatioUserIndexesCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{"schemaname": "unknown", "relname": "unknown", "indexrelname": "unknown"}, value: 0, metricType: dto.MetricType_COUNTER}, + {labels: labelMap{"schemaname": "unknown", "relname": "unknown", "indexrelname": "unknown"}, value: 0, metricType: dto.MetricType_COUNTER}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_statio_user_tables.go b/collector/pg_statio_user_tables.go new file mode 100644 index 000000000..4315fda0a --- /dev/null +++ b/collector/pg_statio_user_tables.go @@ -0,0 +1,222 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + "database/sql" + + "github.com/go-kit/log" + "github.com/prometheus/client_golang/prometheus" +) + +const statioUserTableSubsystem = "statio_user_tables" + +func init() { + registerCollector(statioUserTableSubsystem, defaultEnabled, NewPGStatIOUserTablesCollector) +} + +type PGStatIOUserTablesCollector struct { + log log.Logger +} + +func NewPGStatIOUserTablesCollector(config collectorConfig) (Collector, error) { + return &PGStatIOUserTablesCollector{log: config.logger}, nil +} + +var ( + statioUserTablesHeapBlksRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "heap_blocks_read"), + "Number of disk blocks read from this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesHeapBlksHit = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "heap_blocks_hit"), + "Number of buffer hits in this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesIdxBlksRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "idx_blocks_read"), + "Number of disk blocks read from all indexes on this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesIdxBlksHit = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "idx_blocks_hit"), + "Number of buffer hits in all indexes on this table", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesToastBlksRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "toast_blocks_read"), + "Number of disk blocks read from this table's TOAST table (if any)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesToastBlksHit = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "toast_blocks_hit"), + "Number of buffer hits in this table's TOAST table (if any)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesTidxBlksRead = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "tidx_blocks_read"), + "Number of disk blocks read from this table's TOAST table indexes (if any)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + statioUserTablesTidxBlksHit = prometheus.NewDesc( + prometheus.BuildFQName(namespace, statioUserTableSubsystem, "tidx_blocks_hit"), + "Number of buffer hits in this table's TOAST table indexes (if any)", + []string{"datname", "schemaname", "relname"}, + prometheus.Labels{}, + ) + + statioUserTablesQuery = `SELECT + current_database() datname, + schemaname, + relname, + heap_blks_read, + heap_blks_hit, + idx_blks_read, + idx_blks_hit, + toast_blks_read, + toast_blks_hit, + tidx_blks_read, + tidx_blks_hit + FROM pg_statio_user_tables` +) + +func (PGStatIOUserTablesCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + rows, err := db.QueryContext(ctx, + statioUserTablesQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var datname, schemaname, relname sql.NullString + var heapBlksRead, heapBlksHit, idxBlksRead, idxBlksHit, toastBlksRead, toastBlksHit, tidxBlksRead, tidxBlksHit sql.NullInt64 + + if err := rows.Scan(&datname, &schemaname, &relname, &heapBlksRead, &heapBlksHit, &idxBlksRead, &idxBlksHit, &toastBlksRead, &toastBlksHit, &tidxBlksRead, &tidxBlksHit); err != nil { + return err + } + datnameLabel := "unknown" + if datname.Valid { + datnameLabel = datname.String + } + schemanameLabel := "unknown" + if schemaname.Valid { + schemanameLabel = schemaname.String + } + relnameLabel := "unknown" + if relname.Valid { + relnameLabel = relname.String + } + + heapBlksReadMetric := 0.0 + if heapBlksRead.Valid { + heapBlksReadMetric = float64(heapBlksRead.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesHeapBlksRead, + prometheus.CounterValue, + heapBlksReadMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + heapBlksHitMetric := 0.0 + if heapBlksHit.Valid { + heapBlksHitMetric = float64(heapBlksHit.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesHeapBlksHit, + prometheus.CounterValue, + heapBlksHitMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + idxBlksReadMetric := 0.0 + if idxBlksRead.Valid { + idxBlksReadMetric = float64(idxBlksRead.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesIdxBlksRead, + prometheus.CounterValue, + idxBlksReadMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + idxBlksHitMetric := 0.0 + if idxBlksHit.Valid { + idxBlksHitMetric = float64(idxBlksHit.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesIdxBlksHit, + prometheus.CounterValue, + idxBlksHitMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + toastBlksReadMetric := 0.0 + if toastBlksRead.Valid { + toastBlksReadMetric = float64(toastBlksRead.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesToastBlksRead, + prometheus.CounterValue, + toastBlksReadMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + toastBlksHitMetric := 0.0 + if toastBlksHit.Valid { + toastBlksHitMetric = float64(toastBlksHit.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesToastBlksHit, + prometheus.CounterValue, + toastBlksHitMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + tidxBlksReadMetric := 0.0 + if tidxBlksRead.Valid { + tidxBlksReadMetric = float64(tidxBlksRead.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesTidxBlksRead, + prometheus.CounterValue, + tidxBlksReadMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + + tidxBlksHitMetric := 0.0 + if tidxBlksHit.Valid { + tidxBlksHitMetric = float64(tidxBlksHit.Int64) + } + ch <- prometheus.MustNewConstMetric( + statioUserTablesTidxBlksHit, + prometheus.CounterValue, + tidxBlksHitMetric, + datnameLabel, schemanameLabel, relnameLabel, + ) + } + return rows.Err() +} diff --git a/collector/pg_statio_user_tables_test.go b/collector/pg_statio_user_tables_test.go new file mode 100644 index 000000000..c7304a38c --- /dev/null +++ b/collector/pg_statio_user_tables_test.go @@ -0,0 +1,157 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGStatIOUserTablesCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datname", + "schemaname", + "relname", + "heap_blks_read", + "heap_blks_hit", + "idx_blks_read", + "idx_blks_hit", + "toast_blks_read", + "toast_blks_hit", + "tidx_blks_read", + "tidx_blks_hit", + } + rows := sqlmock.NewRows(columns). + AddRow("postgres", + "public", + "a_table", + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8) + mock.ExpectQuery(sanitizeQuery(statioUserTablesQuery)).WillReturnRows(rows) + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatIOUserTablesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatIOUserTablesCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 1}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 2}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 3}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 4}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 5}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 6}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 7}, + {labels: labelMap{"datname": "postgres", "schemaname": "public", "relname": "a_table"}, metricType: dto.MetricType_COUNTER, value: 8}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} + +func TestPGStatIOUserTablesCollectorNullValues(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{ + "datname", + "schemaname", + "relname", + "heap_blks_read", + "heap_blks_hit", + "idx_blks_read", + "idx_blks_hit", + "toast_blks_read", + "toast_blks_hit", + "tidx_blks_read", + "tidx_blks_hit", + } + rows := sqlmock.NewRows(columns). + AddRow(nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil, + nil) + mock.ExpectQuery(sanitizeQuery(statioUserTablesQuery)).WillReturnRows(rows) + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGStatIOUserTablesCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGStatIOUserTablesCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + {labels: labelMap{"datname": "unknown", "schemaname": "unknown", "relname": "unknown"}, metricType: dto.MetricType_COUNTER, value: 0}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_wal.go b/collector/pg_wal.go new file mode 100644 index 000000000..afa8fcef6 --- /dev/null +++ b/collector/pg_wal.go @@ -0,0 +1,84 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + + "github.com/prometheus/client_golang/prometheus" +) + +const walSubsystem = "wal" + +func init() { + registerCollector(walSubsystem, defaultEnabled, NewPGWALCollector) +} + +type PGWALCollector struct { +} + +func NewPGWALCollector(config collectorConfig) (Collector, error) { + return &PGWALCollector{}, nil +} + +var ( + pgWALSegments = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + walSubsystem, + "segments", + ), + "Number of WAL segments", + []string{}, nil, + ) + pgWALSize = prometheus.NewDesc( + prometheus.BuildFQName( + namespace, + walSubsystem, + "size_bytes", + ), + "Total size of WAL segments", + []string{}, nil, + ) + + pgWALQuery = ` + SELECT + COUNT(*) AS segments, + SUM(size) AS size + FROM pg_ls_waldir() + WHERE name ~ '^[0-9A-F]{24}$'` +) + +func (c PGWALCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + row := db.QueryRowContext(ctx, + pgWALQuery, + ) + + var segments uint64 + var size uint64 + err := row.Scan(&segments, &size) + if err != nil { + return err + } + ch <- prometheus.MustNewConstMetric( + pgWALSegments, + prometheus.GaugeValue, float64(segments), + ) + ch <- prometheus.MustNewConstMetric( + pgWALSize, + prometheus.GaugeValue, float64(size), + ) + return nil +} diff --git a/collector/pg_wal_test.go b/collector/pg_wal_test.go new file mode 100644 index 000000000..745105a13 --- /dev/null +++ b/collector/pg_wal_test.go @@ -0,0 +1,63 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPgWALCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + + inst := &instance{db: db} + + columns := []string{"segments", "size"} + rows := sqlmock.NewRows(columns). + AddRow(47, 788529152) + mock.ExpectQuery(sanitizeQuery(pgWALQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGWALCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGWALCollector.Update: %s", err) + } + }() + + expected := []MetricResult{ + {labels: labelMap{}, value: 47, metricType: dto.MetricType_GAUGE}, + {labels: labelMap{}, value: 788529152, metricType: dto.MetricType_GAUGE}, + } + + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/pg_xlog_location.go b/collector/pg_xlog_location.go new file mode 100644 index 000000000..237204f7d --- /dev/null +++ b/collector/pg_xlog_location.go @@ -0,0 +1,91 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package collector + +import ( + "context" + + "github.com/blang/semver/v4" + "github.com/go-kit/log" + "github.com/go-kit/log/level" + "github.com/prometheus/client_golang/prometheus" +) + +const xlogLocationSubsystem = "xlog_location" + +func init() { + registerCollector(xlogLocationSubsystem, defaultDisabled, NewPGXlogLocationCollector) +} + +type PGXlogLocationCollector struct { + log log.Logger +} + +func NewPGXlogLocationCollector(config collectorConfig) (Collector, error) { + return &PGXlogLocationCollector{log: config.logger}, nil +} + +var ( + xlogLocationBytes = prometheus.NewDesc( + prometheus.BuildFQName(namespace, xlogLocationSubsystem, "bytes"), + "Postgres LSN (log sequence number) being generated on primary or replayed on replica (truncated to low 52 bits)", + []string{}, + prometheus.Labels{}, + ) + + xlogLocationQuery = ` + SELECT CASE + WHEN pg_is_in_recovery() THEN (pg_last_xlog_replay_location() - '0/0') % (2^52)::bigint + ELSE (pg_current_xlog_location() - '0/0') % (2^52)::bigint + END AS bytes + ` +) + +func (c PGXlogLocationCollector) Update(ctx context.Context, instance *instance, ch chan<- prometheus.Metric) error { + db := instance.getDB() + + // xlog was renmaed to WAL in PostgreSQL 10 + // https://wiki.postgresql.org/wiki/New_in_postgres_10#Renaming_of_.22xlog.22_to_.22wal.22_Globally_.28and_location.2Flsn.29 + after10 := instance.version.Compare(semver.MustParse("10.0.0")) + if after10 >= 0 { + level.Warn(c.log).Log("msg", "xlog_location collector is not available on PostgreSQL >= 10.0.0, skipping") + return nil + } + + rows, err := db.QueryContext(ctx, + xlogLocationQuery) + + if err != nil { + return err + } + defer rows.Close() + + for rows.Next() { + var bytes float64 + + if err := rows.Scan(&bytes); err != nil { + return err + } + + ch <- prometheus.MustNewConstMetric( + xlogLocationBytes, + prometheus.GaugeValue, + bytes, + ) + } + if err := rows.Err(); err != nil { + return err + } + return nil +} diff --git a/collector/pg_xlog_location_test.go b/collector/pg_xlog_location_test.go new file mode 100644 index 000000000..561a7df94 --- /dev/null +++ b/collector/pg_xlog_location_test.go @@ -0,0 +1,61 @@ +// Copyright 2023 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package collector + +import ( + "context" + "testing" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/smartystreets/goconvey/convey" +) + +func TestPGXlogLocationCollector(t *testing.T) { + db, mock, err := sqlmock.New() + if err != nil { + t.Fatalf("Error opening a stub db connection: %s", err) + } + defer db.Close() + inst := &instance{db: db} + columns := []string{ + "bytes", + } + rows := sqlmock.NewRows(columns). + AddRow(53401) + + mock.ExpectQuery(sanitizeQuery(xlogLocationQuery)).WillReturnRows(rows) + + ch := make(chan prometheus.Metric) + go func() { + defer close(ch) + c := PGXlogLocationCollector{} + + if err := c.Update(context.Background(), inst, ch); err != nil { + t.Errorf("Error calling PGXlogLocationCollector.Update: %s", err) + } + }() + expected := []MetricResult{ + {labels: labelMap{}, value: 53401, metricType: dto.MetricType_GAUGE}, + } + convey.Convey("Metrics comparison", t, func() { + for _, expect := range expected { + m := readMetric(<-ch) + convey.So(expect, convey.ShouldResemble, m) + } + }) + if err := mock.ExpectationsWereMet(); err != nil { + t.Errorf("there were unfulfilled exceptions: %s", err) + } +} diff --git a/collector/probe.go b/collector/probe.go index 37cb151b9..4c0f0419b 100644 --- a/collector/probe.go +++ b/collector/probe.go @@ -15,10 +15,10 @@ package collector import ( "context" - "database/sql" "sync" "github.com/go-kit/log" + "github.com/go-kit/log/level" "github.com/prometheus-community/postgres_exporter/config" "github.com/prometheus/client_golang/prometheus" ) @@ -27,10 +27,10 @@ type ProbeCollector struct { registry *prometheus.Registry collectors map[string]Collector logger log.Logger - db *sql.DB + instance *instance } -func NewProbeCollector(logger log.Logger, registry *prometheus.Registry, dsn config.DSN) (*ProbeCollector, error) { +func NewProbeCollector(logger log.Logger, excludeDatabases []string, registry *prometheus.Registry, dsn config.DSN) (*ProbeCollector, error) { collectors := make(map[string]Collector) initiatedCollectorsMtx.Lock() defer initiatedCollectorsMtx.Unlock() @@ -45,7 +45,11 @@ func NewProbeCollector(logger log.Logger, registry *prometheus.Registry, dsn con if collector, ok := initiatedCollectors[key]; ok { collectors[key] = collector } else { - collector, err := factories[key](log.With(logger, "collector", key)) + collector, err := factories[key]( + collectorConfig{ + logger: log.With(logger, "collector", key), + excludeDatabases: excludeDatabases, + }) if err != nil { return nil, err } @@ -54,18 +58,16 @@ func NewProbeCollector(logger log.Logger, registry *prometheus.Registry, dsn con } } - db, err := sql.Open("postgres", dsn.GetConnectionString()) + instance, err := newInstance(dsn.GetConnectionString()) if err != nil { return nil, err } - db.SetMaxOpenConns(1) - db.SetMaxIdleConns(1) return &ProbeCollector{ registry: registry, collectors: collectors, logger: logger, - db: db, + instance: instance, }, nil } @@ -73,13 +75,25 @@ func (pc *ProbeCollector) Describe(ch chan<- *prometheus.Desc) { } func (pc *ProbeCollector) Collect(ch chan<- prometheus.Metric) { + // Set up the database connection for the collector. + err := pc.instance.setup() + if err != nil { + level.Error(pc.logger).Log("msg", "Error opening connection to database", "err", err) + return + } + defer pc.instance.Close() + wg := sync.WaitGroup{} wg.Add(len(pc.collectors)) for name, c := range pc.collectors { go func(name string, c Collector) { - execute(context.TODO(), name, c, pc.db, ch, pc.logger) + execute(context.TODO(), name, c, pc.instance, ch, pc.logger) wg.Done() }(name, c) } wg.Wait() } + +func (pc *ProbeCollector) Close() error { + return pc.instance.Close() +} diff --git a/config/config.go b/config/config.go index 9e514f413..f67969725 100644 --- a/config/config.go +++ b/config/config.go @@ -54,18 +54,18 @@ type UserPass struct { Password string `yaml:"password"` } -type ConfigHandler struct { +type Handler struct { sync.RWMutex Config *Config } -func (ch *ConfigHandler) GetConfig() *Config { +func (ch *Handler) GetConfig() *Config { ch.RLock() defer ch.RUnlock() return ch.Config } -func (ch *ConfigHandler) ReloadConfig(f string, logger log.Logger) error { +func (ch *Handler) ReloadConfig(f string, logger log.Logger) error { config := &Config{} var err error defer func() { diff --git a/config/config_test.go b/config/config_test.go index 63b932adb..d5d23d3ba 100644 --- a/config/config_test.go +++ b/config/config_test.go @@ -18,7 +18,7 @@ import ( ) func TestLoadConfig(t *testing.T) { - ch := &ConfigHandler{ + ch := &Handler{ Config: &Config{}, } @@ -29,7 +29,7 @@ func TestLoadConfig(t *testing.T) { } func TestLoadBadConfigs(t *testing.T) { - ch := &ConfigHandler{ + ch := &Handler{ Config: &Config{}, } diff --git a/config/dsn.go b/config/dsn.go index 78d798d5f..168d00d62 100644 --- a/config/dsn.go +++ b/config/dsn.go @@ -65,7 +65,7 @@ func (d DSN) GetConnectionString() string { // dsnFromString parses a connection string into a dsn. It will attempt to parse the string as // a URL and as a set of key=value pairs. If both attempts fail, dsnFromString will return an error. func dsnFromString(in string) (DSN, error) { - if strings.HasPrefix(in, "postgresql://") { + if strings.HasPrefix(in, "postgresql://") || strings.HasPrefix(in, "postgres://") { return dsnFromURL(in) } diff --git a/config/dsn_test.go b/config/dsn_test.go index 637a3568e..68340cd09 100644 --- a/config/dsn_test.go +++ b/config/dsn_test.go @@ -186,6 +186,19 @@ func Test_dsnFromString(t *testing.T) { }, wantErr: false, }, + { + name: "Alternative URL prefix", + input: "postgres://user:s3cret@host.example.com:5432/tsdb?user=postgres", + want: DSN{ + scheme: "postgres", + host: "host.example.com:5432", + path: "/tsdb", + query: url.Values{}, + username: "user", + password: "s3cret", + }, + wantErr: false, + }, { name: "URL with user and password in query string", input: "postgresql://host.example.com:5432/tsdb?user=postgres&password=s3cr3t", diff --git a/go.mod b/go.mod index e36c211b6..963ef770b 100644 --- a/go.mod +++ b/go.mod @@ -1,40 +1,47 @@ module github.com/prometheus-community/postgres_exporter -go 1.20 +go 1.19 require ( + github.com/DATA-DOG/go-sqlmock v1.5.0 + github.com/alecthomas/kingpin/v2 v2.3.2 github.com/blang/semver/v4 v4.0.0 github.com/go-kit/log v0.2.1 - github.com/lib/pq v1.10.7 - github.com/prometheus/client_golang v1.13.0 - github.com/prometheus/client_model v0.2.0 - github.com/prometheus/common v0.37.0 - github.com/prometheus/exporter-toolkit v0.7.1 - gopkg.in/alecthomas/kingpin.v2 v2.2.6 + github.com/lib/pq v1.10.9 + github.com/prometheus/client_golang v1.18.0 + github.com/prometheus/client_model v0.5.0 + github.com/prometheus/common v0.45.0 + github.com/prometheus/exporter-toolkit v0.10.0 + github.com/smartystreets/goconvey v1.8.1 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v3 v3.0.1 ) require ( - github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect - github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d // indirect + github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/cespare/xxhash/v2 v2.1.2 // indirect + github.com/cespare/xxhash/v2 v2.2.0 // indirect + github.com/coreos/go-systemd/v22 v22.5.0 // indirect github.com/go-logfmt/logfmt v0.5.1 // indirect - github.com/golang/protobuf v1.5.2 // indirect + github.com/golang/protobuf v1.5.3 // indirect + github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/jpillora/backoff v1.0.0 // indirect - github.com/kr/pretty v0.2.1 // indirect - github.com/kr/text v0.1.0 // indirect - github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect + github.com/jtolds/gls v4.20.0+incompatible // indirect + github.com/kr/pretty v0.3.1 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f // indirect - github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/procfs v0.8.0 // indirect - golang.org/x/crypto v0.1.0 // indirect - golang.org/x/net v0.7.0 // indirect - golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b // indirect - golang.org/x/sys v0.5.0 // indirect - golang.org/x/text v0.7.0 // indirect - google.golang.org/appengine v1.6.6 // indirect - google.golang.org/protobuf v1.28.1 // indirect + github.com/prometheus/procfs v0.12.0 // indirect + github.com/rogpeppe/go-internal v1.10.0 // indirect + github.com/smarty/assertions v1.15.0 // indirect + github.com/xhit/go-str2duration/v2 v2.1.0 // indirect + golang.org/x/crypto v0.17.0 // indirect + golang.org/x/net v0.17.0 // indirect + golang.org/x/oauth2 v0.12.0 // indirect + golang.org/x/sync v0.3.0 // indirect + golang.org/x/sys v0.15.0 // indirect + golang.org/x/text v0.14.0 // indirect + google.golang.org/appengine v1.6.7 // indirect + google.golang.org/protobuf v1.31.0 // indirect ) diff --git a/go.sum b/go.sum index 0a823ed3a..5322464b6 100644 --- a/go.sum +++ b/go.sum @@ -1,513 +1,105 @@ -cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= -cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= -cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= -cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= -cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= -cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= -cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= -cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= -cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= -cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= -cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= -cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= -cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= -cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= -cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= -cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= -cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= -cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= -cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= -cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= -cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= -cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= -cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= -cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= -cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= -cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= -cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= -cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= -cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= -cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= -dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= -github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= -github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 h1:JYp7IbQjafoB+tBA3gMyHYHrpOtNuDiK/uB5uXxq5wM= -github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d h1:UQZhZ2O0vMHr2cI+DC1Mbh0TJxzA3RcLoMsFw+aXw7E= -github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= -github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/DATA-DOG/go-sqlmock v1.5.0 h1:Shsta01QNfFxHCfpW6YH2STWB0MudeXXEWMr20OEh60= +github.com/DATA-DOG/go-sqlmock v1.5.0/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM= +github.com/alecthomas/kingpin/v2 v2.3.2 h1:H0aULhgmSzN8xQ3nX1uxtdlTHYoPLu5AhHxWrKI6ocU= +github.com/alecthomas/kingpin/v2 v2.3.2/go.mod h1:0gyi0zQnjuFk8xrkNKamJoyUo382HRL7ATRpFZCw6tE= +github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc= +github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8VM1Kc9e19TLln2VL61YJF0x1XFtfdL4JdbSyE= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= -github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= -github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= -github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= +github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= +github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs= +github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= -github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= -github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= -github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= -github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= github.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU= github.com/go-kit/log v0.2.1/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= -github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= -github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= -github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= github.com/go-logfmt/logfmt v0.5.1 h1:otpy5pqBCBZ1ng9RQ0dPu4PN7ba75Y/aA+UpowDyNVA= github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs= -github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= -github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= -github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= -github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= -github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= -github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= -github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= -github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= -github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= -github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= -github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= -github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= -github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg= -github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= -github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= -github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= -github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= -github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= +github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= +github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= -github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= -github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= -github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= -github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= -github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= -github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= +github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= +github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= -github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= -github.com/lib/pq v1.10.7 h1:p7ZhMD+KsSRozJr34udlUrhboJwWAgCg34+/ZZNvZZw= -github.com/lib/pq v1.10.7/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o= -github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= -github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= -github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= -github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw= +github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o= +github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvlsiIGKtc+UG6U5vzxaoagmhXfyg= +github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f h1:KUppIJq7/+SVif2QVs3tOP0zanoHgBEVAwHxUSIzRqU= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= -github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= -github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= -github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= -github.com/prometheus/client_golang v1.13.0 h1:b71QUfeo5M8gq2+evJdTPfZhYMAU0uKPkyPJ7TPsloU= -github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ= -github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= -github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= -github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= -github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.29.0/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= -github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= -github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE= -github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA= -github.com/prometheus/exporter-toolkit v0.7.1 h1:c6RXaK8xBVercEeUQ4tRNL8UGWzDHfvj9dseo1FcK1Y= -github.com/prometheus/exporter-toolkit v0.7.1/go.mod h1:ZUBIj498ePooX9t/2xtDjeQYwvRpiPP2lh5u4iblj2g= -github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= -github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= -github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo= -github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4= -github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= -github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= -github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= +github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk= +github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA= +github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw= +github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI= +github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM= +github.com/prometheus/common v0.45.0/go.mod h1:YJmSTw9BoKxJplESWWxlbyttQR4uaEcGyv9MZjVOJsY= +github.com/prometheus/exporter-toolkit v0.10.0 h1:yOAzZTi4M22ZzVxD+fhy1URTuNRj/36uQJJ5S8IPza8= +github.com/prometheus/exporter-toolkit v0.10.0/go.mod h1:+sVFzuvV5JDyw+Ih6p3zFxZNVnKQa3x5qPmDSiPu4ZY= +github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo= +github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo= +github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY= +github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec= +github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY= +github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= -github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= -go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= -go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8= +github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= +github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.1.0 h1:MDRAIl0xIo9Io2xV565hzXHw3zVseKrJKodhohM5CjU= -golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw= -golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= -golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= -golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= -golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= -golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= -golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= -golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= -golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= -golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= -golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= -golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= -golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= -golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= -golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= -golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= -golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= -golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= -golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k= +golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= -golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= -golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= -golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b h1:clP8eMhB30EHdc0bd2Twtq6kgU7yl5ub2cQLSdrv1Dg= -golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= +golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= +golang.org/x/oauth2 v0.12.0 h1:smVPGxink+n1ZI5pkQa8y6fZT0RW0MgCO5bFpepy4B4= +golang.org/x/oauth2 v0.12.0/go.mod h1:A74bZ3aGXgCY0qaIC9Ahg6Lglin4AMAco8cIv9baba4= +golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E= +golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= -golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= +golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= -golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= -golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= -golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= -golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= -golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= -google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= -google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= -google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= -google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= -google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= -google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= -google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc= -google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= -google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= -google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= -google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= -google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= -google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= -google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= -google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= -google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= -google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= -google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= -google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= -google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= -google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= -google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= -google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= -google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= -google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= -google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= -google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= -google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= +google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= +google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w= -google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= -gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc= -gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= +google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= -gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= -honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= -rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= -rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= diff --git a/postgres_exporter.rc b/postgres_exporter.rc index d7c554689..b7a59a960 100644 --- a/postgres_exporter.rc +++ b/postgres_exporter.rc @@ -56,7 +56,8 @@ postgres_exporter_data_source_name="postgresql://${postgres_exporter_pg_user}:${ pidfile=/var/run/postgres_exporter.pid command="/usr/sbin/daemon" procname="/usr/local/bin/postgres_exporter" -command_args="-p ${pidfile} /usr/bin/env DATA_SOURCE_NAME="${postgres_exporter_data_source_name}" ${procname} \ +command_args="-f -p ${pidfile} -T ${name} \ + /usr/bin/env DATA_SOURCE_NAME="${postgres_exporter_data_source_name}" ${procname} \ --web.listen-address=${postgres_exporter_listen_address} \ ${postgres_exporter_args}" diff --git a/postgres_mixin/dashboards/postgres-overview.json b/postgres_mixin/dashboards/postgres-overview.json index 5bea49262..9bf41be6a 100644 --- a/postgres_mixin/dashboards/postgres-overview.json +++ b/postgres_mixin/dashboards/postgres-overview.json @@ -25,7 +25,7 @@ "bars": false, "dashLength": 10, "dashes": false, - "datasource": "Postgres Overview", + "datasource": "$datasource", "editable": true, "error": false, "fieldConfig": { @@ -77,7 +77,7 @@ { "alias": "fetched", "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_fetched{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_tup_fetched{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -131,7 +131,7 @@ { "alias": "fetched", "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_returned{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_tup_returned{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -185,7 +185,7 @@ { "alias": "fetched", "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_inserted{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_tup_inserted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -239,7 +239,7 @@ { "alias": "fetched", "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_updated{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_tup_updated{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -293,7 +293,7 @@ { "alias": "fetched", "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_tup_deleted{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_tup_deleted{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -396,7 +396,7 @@ "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], - "datasource": "Postgres Overview", + "datasource": "$datasource", "decimals": 0, "editable": true, "error": false, @@ -460,7 +460,7 @@ "targets": [ { "dsType": "prometheus", - "expr": "sum(irate(pg_stat_database_xact_commit{datname=~\"$db\",instance=~\"$instance\"}[5m])) + sum(irate(pg_stat_database_xact_rollback{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(irate(pg_stat_database_xact_commit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum(irate(pg_stat_database_xact_rollback{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -530,7 +530,7 @@ "bars": false, "dashLength": 10, "dashes": false, - "datasource": "Postgres Overview", + "datasource": "$datasource", "decimals": 1, "editable": true, "error": false, @@ -584,7 +584,7 @@ { "alias": "Buffers Allocated", "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_alloc_total{instance='$instance'}[5m])", + "expr": "irate(pg_stat_bgwriter_buffers_alloc{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -636,7 +636,7 @@ { "alias": "Buffers Allocated", "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_backend_fsync_total{instance='$instance'}[5m])", + "expr": "irate(pg_stat_bgwriter_buffers_backend_fsync{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -688,7 +688,7 @@ { "alias": "Buffers Allocated", "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_backend_total{instance='$instance'}[5m])", + "expr": "irate(pg_stat_bgwriter_buffers_backend{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -740,7 +740,7 @@ { "alias": "Buffers Allocated", "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_clean_total{instance='$instance'}[5m])", + "expr": "irate(pg_stat_bgwriter_buffers_clean{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -792,7 +792,7 @@ { "alias": "Buffers Allocated", "dsType": "prometheus", - "expr": "irate(pg_stat_bgwriter_buffers_checkpoint_total{instance='$instance'}[5m])", + "expr": "irate(pg_stat_bgwriter_buffers_checkpoint{job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])", "format": "time_series", "groupBy": [ { @@ -889,7 +889,7 @@ "bars": false, "dashLength": 10, "dashes": false, - "datasource": "Postgres Overview", + "datasource": "$datasource", "editable": true, "error": false, "fieldConfig": { @@ -939,7 +939,7 @@ { "alias": "conflicts", "dsType": "prometheus", - "expr": "sum(rate(pg_stat_database_deadlocks{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(rate(pg_stat_database_deadlocks{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -991,7 +991,7 @@ { "alias": "deadlocks", "dsType": "prometheus", - "expr": "sum(rate(pg_stat_database_conflicts{datname=~\"$db\",instance=~\"$instance\"}[5m]))", + "expr": "sum(rate(pg_stat_database_conflicts{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval]))", "format": "time_series", "groupBy": [ { @@ -1088,7 +1088,7 @@ "bars": false, "dashLength": 10, "dashes": false, - "datasource": "Postgres Overview", + "datasource": "$datasource", "editable": true, "error": false, "fieldConfig": { @@ -1136,10 +1136,10 @@ "steppedLine": false, "targets": [ { - "expr": "sum(pg_stat_database_blks_hit{datname=~\"$db\",instance=~\"$instance\"}) / (sum(pg_stat_database_blks_hit{datname=~\"$db\",instance=~\"$instance\"}) + sum(pg_stat_database_blks_read{datname=~\"$db\",instance=~\"$instance\"}))", + "expr": "sum by (datname) (rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) / (sum by (datname)(rate(pg_stat_database_blks_hit{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])) + sum by (datname)(rate(pg_stat_database_blks_read{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}[$__rate_interval])))", "format": "time_series", "intervalFactor": 2, - "legendFormat": "cache hit rate", + "legendFormat": "{{datname}} - cache hit rate", "refId": "A", "step": 240 } @@ -1191,7 +1191,7 @@ "bars": false, "dashLength": 10, "dashes": false, - "datasource": "Postgres Overview", + "datasource": "$datasource", "editable": true, "error": false, "fieldConfig": { @@ -1239,10 +1239,10 @@ "steppedLine": false, "targets": [ { - "expr": "pg_stat_database_numbackends{datname=~\"$db\",instance=~\"$instance\"}", + "expr": "pg_stat_database_numbackends{datname=~\"$db\",job=~\"$job\",instance=~\"$instance\"}", "format": "time_series", "intervalFactor": 2, - "legendFormat": "{{__name__}}", + "legendFormat": "{{datname}} - {{__name__}}", "refId": "A", "step": 240 } @@ -1299,24 +1299,32 @@ "templating": { "list": [ { - "allValue": ".*", - "current": { - "selected": false, - "text": "All", - "value": "$__all" - }, - "datasource": "Postgres Overview", - "definition": "", "hide": 0, - "includeAll": true, - "label": null, + "includeAll": false, + "label": "Data Source", "multi": false, - "name": "instance", + "name": "datasource", "options": [], - "query": "label_values(up{job=~\"postgres.*\"},instance)", + "query": "prometheus", "refresh": 1, "regex": "", "skipUrlSync": false, + "type": "datasource" + }, + { + "allValue": ".+", + "datasource": "$datasource", + "definition": "label_values(pg_up, job)", + "hide": 0, + "includeAll": true, + "label": "job", + "multi": true, + "name": "job", + "options": [], + "query": "label_values(pg_up, job)", + "refresh": 0, + "regex": "", + "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", "tags": [], @@ -1325,21 +1333,16 @@ "useTags": false }, { - "allValue": ".*", - "current": { - "selected": false, - "text": "All", - "value": "$__all" - }, - "datasource": "Postgres Overview", - "definition": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", + "allValue": ".+", + "datasource": "$datasource", + "definition": "", "hide": 0, "includeAll": true, - "label": "db", - "multi": false, - "name": "db", + "label": "instance", + "multi": true, + "name": "instance", "options": [], - "query": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", + "query": "label_values(up{job=~\"$job\"},instance)", "refresh": 1, "regex": "", "skipUrlSync": false, @@ -1351,48 +1354,19 @@ "useTags": false }, { - "current": { - "selected": false, - "text": "Postgres Overview", - "value": "Postgres Overview" - }, + "allValue": ".+", + "datasource": "$datasource", + "definition": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", "hide": 0, - "includeAll": false, - "label": "datasource", + "includeAll": true, + "label": "db", "multi": false, - "name": "datasource", + "name": "db", "options": [], - "query": "prometheus", + "query": "label_values(pg_stat_database_tup_fetched{instance=~\"$instance\",datname!~\"template.*|postgres\"},datname)", "refresh": 1, "regex": "", "skipUrlSync": false, - "type": "datasource" - }, - { - "allValue": null, - "current": { - "selected": true, - "text": "postgres", - "value": "postgres" - }, - "datasource": "$datasource", - "definition": "label_values(pg_up, job)", - "hide": 0, - "includeAll": false, - "label": "job", - "multi": false, - "name": "job", - "options": [ - { - "selected": true, - "text": "postgres", - "value": "postgres" - } - ], - "query": "label_values(pg_up, job)", - "refresh": 0, - "regex": "", - "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", "tags": [], diff --git a/queries.yaml b/queries.yaml index 6f2008cbe..189ce0866 100644 --- a/queries.yaml +++ b/queries.yaml @@ -1,263 +1,2 @@ -pg_replication: - query: "SELECT CASE WHEN NOT pg_is_in_recovery() THEN 0 ELSE GREATEST (0, EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) END AS lag" - master: true - metrics: - - lag: - usage: "GAUGE" - description: "Replication lag behind master in seconds" - -pg_postmaster: - query: "SELECT pg_postmaster_start_time as start_time_seconds from pg_postmaster_start_time()" - master: true - metrics: - - start_time_seconds: - usage: "GAUGE" - description: "Time at which postmaster started" - -pg_stat_user_tables: - query: | - SELECT - current_database() datname, - schemaname, - relname, - seq_scan, - seq_tup_read, - idx_scan, - idx_tup_fetch, - n_tup_ins, - n_tup_upd, - n_tup_del, - n_tup_hot_upd, - n_live_tup, - n_dead_tup, - n_mod_since_analyze, - COALESCE(last_vacuum, '1970-01-01Z') as last_vacuum, - COALESCE(last_autovacuum, '1970-01-01Z') as last_autovacuum, - COALESCE(last_analyze, '1970-01-01Z') as last_analyze, - COALESCE(last_autoanalyze, '1970-01-01Z') as last_autoanalyze, - vacuum_count, - autovacuum_count, - analyze_count, - autoanalyze_count - FROM - pg_stat_user_tables - metrics: - - datname: - usage: "LABEL" - description: "Name of current database" - - schemaname: - usage: "LABEL" - description: "Name of the schema that this table is in" - - relname: - usage: "LABEL" - description: "Name of this table" - - seq_scan: - usage: "COUNTER" - description: "Number of sequential scans initiated on this table" - - seq_tup_read: - usage: "COUNTER" - description: "Number of live rows fetched by sequential scans" - - idx_scan: - usage: "COUNTER" - description: "Number of index scans initiated on this table" - - idx_tup_fetch: - usage: "COUNTER" - description: "Number of live rows fetched by index scans" - - n_tup_ins: - usage: "COUNTER" - description: "Number of rows inserted" - - n_tup_upd: - usage: "COUNTER" - description: "Number of rows updated" - - n_tup_del: - usage: "COUNTER" - description: "Number of rows deleted" - - n_tup_hot_upd: - usage: "COUNTER" - description: "Number of rows HOT updated (i.e., with no separate index update required)" - - n_live_tup: - usage: "GAUGE" - description: "Estimated number of live rows" - - n_dead_tup: - usage: "GAUGE" - description: "Estimated number of dead rows" - - n_mod_since_analyze: - usage: "GAUGE" - description: "Estimated number of rows changed since last analyze" - - last_vacuum: - usage: "GAUGE" - description: "Last time at which this table was manually vacuumed (not counting VACUUM FULL)" - - last_autovacuum: - usage: "GAUGE" - description: "Last time at which this table was vacuumed by the autovacuum daemon" - - last_analyze: - usage: "GAUGE" - description: "Last time at which this table was manually analyzed" - - last_autoanalyze: - usage: "GAUGE" - description: "Last time at which this table was analyzed by the autovacuum daemon" - - vacuum_count: - usage: "COUNTER" - description: "Number of times this table has been manually vacuumed (not counting VACUUM FULL)" - - autovacuum_count: - usage: "COUNTER" - description: "Number of times this table has been vacuumed by the autovacuum daemon" - - analyze_count: - usage: "COUNTER" - description: "Number of times this table has been manually analyzed" - - autoanalyze_count: - usage: "COUNTER" - description: "Number of times this table has been analyzed by the autovacuum daemon" - -pg_statio_user_tables: - query: "SELECT current_database() datname, schemaname, relname, heap_blks_read, heap_blks_hit, idx_blks_read, idx_blks_hit, toast_blks_read, toast_blks_hit, tidx_blks_read, tidx_blks_hit FROM pg_statio_user_tables" - metrics: - - datname: - usage: "LABEL" - description: "Name of current database" - - schemaname: - usage: "LABEL" - description: "Name of the schema that this table is in" - - relname: - usage: "LABEL" - description: "Name of this table" - - heap_blks_read: - usage: "COUNTER" - description: "Number of disk blocks read from this table" - - heap_blks_hit: - usage: "COUNTER" - description: "Number of buffer hits in this table" - - idx_blks_read: - usage: "COUNTER" - description: "Number of disk blocks read from all indexes on this table" - - idx_blks_hit: - usage: "COUNTER" - description: "Number of buffer hits in all indexes on this table" - - toast_blks_read: - usage: "COUNTER" - description: "Number of disk blocks read from this table's TOAST table (if any)" - - toast_blks_hit: - usage: "COUNTER" - description: "Number of buffer hits in this table's TOAST table (if any)" - - tidx_blks_read: - usage: "COUNTER" - description: "Number of disk blocks read from this table's TOAST table indexes (if any)" - - tidx_blks_hit: - usage: "COUNTER" - description: "Number of buffer hits in this table's TOAST table indexes (if any)" - -# WARNING: This set of metrics can be very expensive on a busy server as every unique query executed will create an additional time series -pg_stat_statements: - query: "SELECT t2.rolname, t3.datname, queryid, calls, total_time / 1000 as total_time_seconds, min_time / 1000 as min_time_seconds, max_time / 1000 as max_time_seconds, mean_time / 1000 as mean_time_seconds, stddev_time / 1000 as stddev_time_seconds, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time / 1000 as blk_read_time_seconds, blk_write_time / 1000 as blk_write_time_seconds FROM pg_stat_statements t1 JOIN pg_roles t2 ON (t1.userid=t2.oid) JOIN pg_database t3 ON (t1.dbid=t3.oid) WHERE t2.rolname != 'rdsadmin'" - master: true - metrics: - - rolname: - usage: "LABEL" - description: "Name of user" - - datname: - usage: "LABEL" - description: "Name of database" - - queryid: - usage: "LABEL" - description: "Query ID" - - calls: - usage: "COUNTER" - description: "Number of times executed" - - total_time_seconds: - usage: "COUNTER" - description: "Total time spent in the statement, in milliseconds" - - min_time_seconds: - usage: "GAUGE" - description: "Minimum time spent in the statement, in milliseconds" - - max_time_seconds: - usage: "GAUGE" - description: "Maximum time spent in the statement, in milliseconds" - - mean_time_seconds: - usage: "GAUGE" - description: "Mean time spent in the statement, in milliseconds" - - stddev_time_seconds: - usage: "GAUGE" - description: "Population standard deviation of time spent in the statement, in milliseconds" - - rows: - usage: "COUNTER" - description: "Total number of rows retrieved or affected by the statement" - - shared_blks_hit: - usage: "COUNTER" - description: "Total number of shared block cache hits by the statement" - - shared_blks_read: - usage: "COUNTER" - description: "Total number of shared blocks read by the statement" - - shared_blks_dirtied: - usage: "COUNTER" - description: "Total number of shared blocks dirtied by the statement" - - shared_blks_written: - usage: "COUNTER" - description: "Total number of shared blocks written by the statement" - - local_blks_hit: - usage: "COUNTER" - description: "Total number of local block cache hits by the statement" - - local_blks_read: - usage: "COUNTER" - description: "Total number of local blocks read by the statement" - - local_blks_dirtied: - usage: "COUNTER" - description: "Total number of local blocks dirtied by the statement" - - local_blks_written: - usage: "COUNTER" - description: "Total number of local blocks written by the statement" - - temp_blks_read: - usage: "COUNTER" - description: "Total number of temp blocks read by the statement" - - temp_blks_written: - usage: "COUNTER" - description: "Total number of temp blocks written by the statement" - - blk_read_time_seconds: - usage: "COUNTER" - description: "Total time the statement spent reading blocks, in milliseconds (if track_io_timing is enabled, otherwise zero)" - - blk_write_time_seconds: - usage: "COUNTER" - description: "Total time the statement spent writing blocks, in milliseconds (if track_io_timing is enabled, otherwise zero)" - -pg_process_idle: - query: | - WITH - metrics AS ( - SELECT - application_name, - SUM(EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP - state_change))::bigint)::float AS process_idle_seconds_sum, - COUNT(*) AS process_idle_seconds_count - FROM pg_stat_activity - WHERE state = 'idle' - GROUP BY application_name - ), - buckets AS ( - SELECT - application_name, - le, - SUM( - CASE WHEN EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP - state_change)) <= le - THEN 1 - ELSE 0 - END - )::bigint AS bucket - FROM - pg_stat_activity, - UNNEST(ARRAY[1, 2, 5, 15, 30, 60, 90, 120, 300]) AS le - GROUP BY application_name, le - ORDER BY application_name, le - ) - SELECT - application_name, - process_idle_seconds_sum as seconds_sum, - process_idle_seconds_count as seconds_count, - ARRAY_AGG(le) AS seconds, - ARRAY_AGG(bucket) AS seconds_bucket - FROM metrics JOIN buckets USING (application_name) - GROUP BY 1, 2, 3 - metrics: - - application_name: - usage: "LABEL" - description: "Application Name" - - seconds: - usage: "HISTOGRAM" - description: "Idle time of server processes" +# Adding queries to this file is deprecated +# Example queries have been transformed into collectors. \ No newline at end of file diff --git a/scripts/errcheck_excludes.txt b/scripts/errcheck_excludes.txt deleted file mode 100644 index 58c2051ac..000000000 --- a/scripts/errcheck_excludes.txt +++ /dev/null @@ -1,2 +0,0 @@ -// Never check for logger errors. -(github.com/go-kit/log.Logger).Log