Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add core module for checks on a specific host #420

Merged
merged 4 commits into from
Jun 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "pg-index-health/src/main/resources"]
path = pg-index-health/src/main/resources
[submodule "pg-index-health-core/src/main/resources"]
path = pg-index-health-core/src/main/resources
url = https://github.com/mfvanek/pg-index-health-sql.git
12 changes: 6 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,23 +34,23 @@ Each database structure check starts with an SQL query to the pg_catalog.
### Extend domain model (if needed)

pg-index-health is a [multimodule Gradle](https://docs.gradle.org/current/userguide/multi_project_builds.html) project.
Domain model is located in a [pg-index-health-model](https://github.com/mfvanek/pg-index-health/tree/master/pg-index-health-model).
Domain model is located in a [pg-index-health-model](pg-index-health-model).
All domain classes should be minimalistic and well-defined.
They should include enough information to generate corrective SQL migrations via [pg-index-health-generator](https://github.com/mfvanek/pg-index-health/tree/master/pg-index-health-generator).
They should include enough information to generate corrective SQL migrations via [pg-index-health-generator](pg-index-health-generator).

### Add the code for the new check

#### Execution on a specific host

* Checks on a specific host https://github.com/mfvanek/pg-index-health/tree/master/pg-index-health/src/main/java/io/github/mfvanek/pg/checks/host
* [Checks on a specific host](pg-index-health-core%2Fsrc%2Fmain%2Fjava%2Fio%2Fgithub%2Fmfvanek%2Fpg%2Fchecks%2Fhost)

Implement a new class extending [AbstractCheckOnHost](https://github.com/mfvanek/pg-index-health/blob/master/pg-index-health/src/main/java/io/github/mfvanek/pg/checks/host/AbstractCheckOnHost.java).
Implement a new class extending [AbstractCheckOnHost](pg-index-health-core%2Fsrc%2Fmain%2Fjava%2Fio%2Fgithub%2Fmfvanek%2Fpg%2Fchecks%2Fhost%2FAbstractCheckOnHost.java).

#### Execution on the cluster

* Checks on the cluster https://github.com/mfvanek/pg-index-health/tree/master/pg-index-health/src/main/java/io/github/mfvanek/pg/checks/cluster
* [Checks on the cluster](pg-index-health%2Fsrc%2Fmain%2Fjava%2Fio%2Fgithub%2Fmfvanek%2Fpg%2Fchecks%2Fcluster)

Implement a new class extending [AbstractCheckOnCluster](https://github.com/mfvanek/pg-index-health/blob/master/pg-index-health/src/main/java/io/github/mfvanek/pg/checks/cluster/AbstractCheckOnCluster.java).
Implement a new class extending [AbstractCheckOnCluster](pg-index-health%2Fsrc%2Fmain%2Fjava%2Fio%2Fgithub%2Fmfvanek%2Fpg%2Fchecks%2Fcluster%2FAbstractCheckOnCluster.java).

### Write proper tests

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

| Version | Supported |
|---------| ------------------ |
| 0.11.x | :white_check_mark: |
| 0.12.x | :white_check_mark: |

## Reporting a Vulnerability

Expand Down
1 change: 1 addition & 0 deletions pg-index-health-bom/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ dependencies {
api(project(":pg-index-health-testing"))
api(project(":spring-boot-integration:pg-index-health-test-starter"))
api(project(":pg-index-health-logger"))
api(project(":pg-index-health-core"))
}
}

Expand Down
27 changes: 27 additions & 0 deletions pg-index-health-core/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
plugins {
id("java-library")
id("pg-index-health.java-compilation")
id("pg-index-health.java-conventions")
id("pg-index-health.publish")
}

description = "pg-index-health-core is a Java library for analyzing and maintaining indexes and tables health in PostgreSQL databases on a specific host."

dependencies {
api(project(":pg-index-health-model"))
api(project(":pg-index-health-jdbc-connection"))
implementation(libs.slf4j.api)

testImplementation(project(":pg-index-health-testing"))
testImplementation(testFixtures(project(":pg-index-health-model")))
testImplementation(testFixtures(project(":pg-index-health-jdbc-connection")))
testImplementation("org.junit.jupiter:junit-jupiter-params")
testImplementation(libs.logback.classic)
testImplementation("org.mockito:mockito-core")
testImplementation(libs.postgresql)

testFixturesImplementation(libs.jsr305)
testFixturesImplementation(libs.apache.commons.lang3)
testFixturesImplementation(libs.postgresql)
testFixturesImplementation(project(":pg-index-health-testing"))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
* Copyright (c) 2019-2024. Ivan Vakhrushev and others.
* https://github.com/mfvanek/pg-index-health
*
* This file is a part of "pg-index-health" - a Java library for
* analyzing and maintaining indexes health in PostgreSQL databases.
*
* Licensed under the Apache License 2.0
*/

package io.github.mfvanek.pg.checks.host;

import io.github.mfvanek.pg.common.maintenance.DatabaseCheckOnHost;
import io.github.mfvanek.pg.common.maintenance.Diagnostic;
import io.github.mfvanek.pg.model.PgContext;
import io.github.mfvanek.pg.model.index.IndexWithBloat;
import io.github.mfvanek.pg.support.StatisticsAwareTestBase;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static io.github.mfvanek.pg.support.AbstractCheckOnHostAssert.assertThat;

class IndexesWithBloatCheckOnHostTest extends StatisticsAwareTestBase {

private final DatabaseCheckOnHost<IndexWithBloat> check = new IndexesWithBloatCheckOnHost(getPgConnection());

@Test
void shouldSatisfyContract() {
assertThat(check)
.hasType(IndexWithBloat.class)
.hasDiagnostic(Diagnostic.BLOATED_INDEXES)
.hasHost(getHost());
}

@ParameterizedTest
@ValueSource(strings = {PgContext.DEFAULT_SCHEMA_NAME, "custom"})
void onDatabaseWithThem(final String schemaName) {
executeTestOnDatabase(schemaName, dbp -> dbp.withReferences().withData(), ctx -> {
collectStatistics(schemaName);
Assertions.assertThat(existsStatisticsForTable(schemaName, "accounts"))
.isTrue();

assertThat(check)
.executing(ctx)
.hasSize(3)
.containsExactlyInAnyOrder(
IndexWithBloat.of(ctx.enrichWithSchema("accounts"), ctx.enrichWithSchema("accounts_account_number_key"), 0L, 0L, 0),
IndexWithBloat.of(ctx.enrichWithSchema("accounts"), ctx.enrichWithSchema("accounts_pkey"), 0L, 0L, 0),
IndexWithBloat.of(ctx.enrichWithSchema("clients"), ctx.enrichWithSchema("clients_pkey"), 0L, 0L, 0))
.allMatch(i -> i.getIndexSizeInBytes() > 1L)
.allMatch(i -> i.getBloatSizeInBytes() > 1L && i.getBloatPercentage() >= 14);
});
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright (c) 2019-2024. Ivan Vakhrushev and others.
* https://github.com/mfvanek/pg-index-health
*
* This file is a part of "pg-index-health" - a Java library for
* analyzing and maintaining indexes health in PostgreSQL databases.
*
* Licensed under the Apache License 2.0
*/

package io.github.mfvanek.pg.checks.host;

import io.github.mfvanek.pg.common.maintenance.DatabaseCheckOnHost;
import io.github.mfvanek.pg.common.maintenance.Diagnostic;
import io.github.mfvanek.pg.model.PgContext;
import io.github.mfvanek.pg.model.table.TableWithBloat;
import io.github.mfvanek.pg.support.StatisticsAwareTestBase;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static io.github.mfvanek.pg.support.AbstractCheckOnHostAssert.assertThat;

class TablesWithBloatCheckOnHostTest extends StatisticsAwareTestBase {

private final DatabaseCheckOnHost<TableWithBloat> check = new TablesWithBloatCheckOnHost(getPgConnection());

@Test
void shouldSatisfyContract() {
assertThat(check)
.hasType(TableWithBloat.class)
.hasDiagnostic(Diagnostic.BLOATED_TABLES)
.hasHost(getHost());
}

@ParameterizedTest
@ValueSource(strings = {PgContext.DEFAULT_SCHEMA_NAME, "custom"})
void onDatabaseWithThem(final String schemaName) {
executeTestOnDatabase(schemaName, dbp -> dbp.withReferences().withData(), ctx -> {
collectStatistics(schemaName);
Assertions.assertThat(existsStatisticsForTable(schemaName, "accounts"))
.isTrue();

assertThat(check)
.executing(ctx)
.hasSize(2)
.containsExactlyInAnyOrder(
TableWithBloat.of(ctx.enrichWithSchema("accounts"), 0L, 0L, 0),
TableWithBloat.of(ctx.enrichWithSchema("clients"), 0L, 0L, 0))
.allMatch(t -> t.getTableSizeInBytes() > 0L) // real size doesn't matter
.allMatch(t -> t.getBloatPercentage() == 0 && t.getBloatSizeInBytes() == 0L);
});
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,7 @@ void shouldBeAtLeastTwoChecksAcrossTheCluster() {
assertThat(d.getQueryExecutor()).isNotNull();
assertThat(d.getExecutionTopology()).isNotNull();
})
.filter(d -> d.getExecutionTopology() == Diagnostic.ExecutionTopology.ACROSS_CLUSTER)
.peek(d -> assertThat(d.isAcrossCluster()).isTrue())
.filter(Diagnostic::isAcrossCluster)
.count();
assertThat(countOfChecksAcrossTheCluster).isEqualTo(2);
}
Expand Down
3 changes: 2 additions & 1 deletion pg-index-health-logger/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ description = "pg-index-health-logger is a Java library for collecting and loggi
dependencies {
api(project(":pg-index-health-model"))
api(project(":pg-index-health-jdbc-connection"))
api(project(":pg-index-health-core"))
api(project(":pg-index-health"))
implementation(libs.slf4j.api)

testImplementation(testFixtures(project(":pg-index-health")))
testImplementation(testFixtures(project(":pg-index-health-core")))
testImplementation("org.junit.jupiter:junit-jupiter-params")
testImplementation(libs.logback.classic)
testImplementation("org.mockito:mockito-core")
Expand Down
9 changes: 3 additions & 6 deletions pg-index-health/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,23 @@ plugins {
id("pg-index-health.publish")
}

description = "pg-index-health is a Java library for analyzing and maintaining indexes health in PostgreSQL databases."
description = "pg-index-health is a Java library for analyzing and maintaining indexes and tables health in PostgreSQL databases on all hosts in the cluster."

dependencies {
api(project(":pg-index-health-model"))
api(project(":pg-index-health-jdbc-connection"))
api(project(":pg-index-health-core"))
implementation(libs.slf4j.api)

testImplementation(project(":pg-index-health-testing"))
testImplementation(testFixtures(project(":pg-index-health-model")))
testImplementation(testFixtures(project(":pg-index-health-jdbc-connection")))
testImplementation(testFixtures(project(":pg-index-health-core")))
testImplementation("org.junit.jupiter:junit-jupiter-params")
testImplementation(libs.logback.classic)
testImplementation("org.mockito:mockito-core")
testImplementation(libs.awaitility)
testImplementation(libs.postgresql)

testFixturesImplementation(libs.jsr305)
testFixturesImplementation(libs.apache.commons.lang3)
testFixturesImplementation(libs.postgresql)
testFixturesImplementation(project(":pg-index-health-testing"))
}

tasks {
Expand Down

This file was deleted.

This file was deleted.

1 change: 1 addition & 0 deletions settings.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ include("spring-boot-integration:kotlin-demo-app")
findProject(":spring-boot-integration:kotlin-demo-app")?.name = "kotlin-demo-app"
include("pg-index-health-bom")
include("pg-index-health-logger")
include("pg-index-health-core")
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@ plugins {
id("pg-index-health.pitest")
}

description = "Spring Boot Starter for pg-index-health library"
description = "Spring Boot Starter for pg-index-health-core library"

dependencies {
api(project(":pg-index-health"))
api(project(":pg-index-health-model"))
api(project(":pg-index-health-jdbc-connection"))
api(project(":pg-index-health-core"))
implementation(libs.spring.boot.starter.root)
annotationProcessor(libs.spring.boot.autoconfigure.processor)
annotationProcessor(libs.spring.boot.configuration.processor)
Expand Down