Add --exclude-snapshots option, which makes exclude pg_logical/snapshots #272
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ots dirs. If there are many large file size temporary files or logical replication snapshot files in the database cluster directory, it will take longer to get a full backup. These files are not likely to be used after the restore. Therefore, I have added an option to exclude these files when obtaining full backups.
|
It was pointed out to me that pgsql_tmp is already excluded in the current code, so I'll fixed it. |
In the most recent commit, I added a --exclude-tmpdir option that also excludes snapshots and pgsql_tmp directory, but since pgsql_tmp was already always excluded, I limited it to only excluding snapshots.
harukat
changed the title
harukat
changed the title
shinyaaa
reviewed
Jan 24, 2025
Comment on lines
+215
to
+218
| if (exclude_snapshots) | ||
| { | ||
| pgdata_exclude[i++] = SNAPSHOTS_DIR; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (exclude_snapshots) | |
| { | |
| pgdata_exclude[i++] = SNAPSHOTS_DIR; | |
| } | |
| if (exclude_snapshots) | |
| pgdata_exclude[i++] = SNAPSHOTS_DIR; |
| <li><strong><code>--exclude-snapshots</code></strong> | ||
|
|
||
| <ul> | ||
| <li>一時ファイル用に使われる <code>pgsql_tmp</code> ディレクトリ内と、ロジカルレプリケーションで使われる <code>snapshots</code> ディレクトリ内をバックアップ対象から除外します。</li> |
There was a problem hiding this comment.
Suggested change
| <li>一時ファイル用に使われる <code>pgsql_tmp</code> ディレクトリ内と、ロジカルレプリケーションで使われる <code>snapshots</code> ディレクトリ内をバックアップ対象から除外します。</li> | |
| <li>ロジカルレプリケーションで使われる <code>$PGDATA/pg_logical/snapshots</code> ディレクトリ内をバックアップ対象から除外します。</li> |
| <td>–exclude-snapshots</td> | ||
| <td>EXCLUDE_TMPDIR</td> | ||
| <td>指定可</td> | ||
| <td>ロジカルスナップショットをバックアップから除外</td> |
There was a problem hiding this comment.
I'm not sure "ロジカルスナップショット" is general representation.
Suggested change
| <td>ロジカルスナップショットをバックアップから除外</td> | |
| <td>ロジカルレプリケーションのスナップショットをバックアップから除外</td> |
| <li><strong><code>--exclude-snapshots</code></strong> | ||
|
|
||
| <ul> | ||
| <li>Exclude <code>snapshots</code> directory, which is used for logical replication.</li> |
There was a problem hiding this comment.
Suggested change
| <li>Exclude <code>snapshots</code> directory, which is used for logical replication.</li> | |
| <li>Exclude <code>$PGDATA/pg_logical/snapshots</code> directory which is used for logical replication.</li> |
| <tr> | ||
| <td></td> | ||
| <td>–exclude-snapshots</td> | ||
| <td>EXCLUDE_TMPDIR</td> |
There was a problem hiding this comment.
This env name is confuse us.
Suggested change
| <td>EXCLUDE_TMPDIR</td> | |
| <td>EXCLUDE_SNAPSHOT_DIR</td> |
| @@ -40,6 +40,7 @@ Backup options: | |||
| --keep-srvlog-days=DAY keep serverlog modified in DAY days | |||
| --standby-host=HOSTNAME standby host when taking backup from standby | |||
| --standby-port=PORT standby port when taking backup from standby | |||
| --exclude-snapshots exclude pg_logical/snapshots | |||
There was a problem hiding this comment.
Suggested change
| --exclude-snapshots exclude pg_logical/snapshots | |
| --exclude-snapshots exclude $PGDATA/pg_logical/snapshots |
| @@ -17,6 +17,7 @@ | |||
| const char *PROGRAM_VERSION = "1.3.16"; | |||
| const char *PROGRAM_URL = "http://github.com/ossc-db/pg_rman"; | |||
| const char *PROGRAM_ISSUES = "http://github.com/ossc-db/pg_rman/issues"; | |||
| const char *SNAPSHOTS_DIR = "snapshots"; | |||
There was a problem hiding this comment.
I think it should be below.
Lines 27 to 45 in d1b5c57
| @@ -32,6 +33,7 @@ bool check = false; | |||
|
|
|||
| /* directory configuration */ | |||
| pgBackup current; | |||
| static bool exclude_snapshots = false; | |||
There was a problem hiding this comment.
I think this is backup configuration. For example, directory options are -D, -A, -B, -S, and -G.
| @@ -97,6 +99,7 @@ static pgut_option options[] = | |||
| { 's', 10, "recovery-target-timeline" , &target_tli_string, SOURCE_ENV }, | |||
| { 's', 11, "recovery-target-action" , &target_action , SOURCE_ENV }, | |||
| { 'b', 12, "hard-copy" , &is_hard_copy , SOURCE_ENV }, | |||
| { 'b', 15, "exclude-snapshots", &exclude_snapshots, SOURCE_ENV }, | |||
There was a problem hiding this comment.
I think this is backup configuration.
| @@ -289,6 +296,7 @@ pgut_help(bool details) | |||
| printf(_(" --keep-srvlog-days=DAY keep serverlog modified in DAY days\n")); | |||
| printf(_(" --standby-host=HOSTNAME standby host when taking backup from standby\n")); | |||
| printf(_(" --standby-port=PORT standby port when taking backup from standby\n")); | |||
| printf(_(" --exclude-snapshots exclude pg_logical/snapshots\n")); | |||
There was a problem hiding this comment.
Suggested change
| printf(_(" --exclude-snapshots exclude pg_logical/snapshots\n")); | |
| printf(_(" --exclude-snapshots exclude $PGDATA/pg_logical/snapshots\n")); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If there are many large file size temporary files or logical replication snapshot files in the database cluster directory, it will take longer to get a full backup. These files are not likely to be used after the restore. Therefore, I have added an option to exclude these files when obtaining full backups.
Since pg_basebackup always excludes pgsql_tmp, it is possible to always exclude pgsql_tmp and make only snapshots optional. Please let us know what you think.