Downloads
F.43. pgpro_bindump — a replication protocol module for backup and restore
Prev UpAppendix F. Additional Supplied Modules and Extensions Shipped in postgrespro-std-16-contribHome Next

F.43. pgpro_bindump — a replication protocol module for backup and restore #

F.43.1. Architecture
F.43.2. Limitations and Requirements
F.43.3. Installation and Setup
F.43.4. Replication Commands
F.43.5. Authors

pgpro_bindump is a module designed to implement additional commands for the Postgres Pro replication protocol, specifically for use with the pg_probackup3 utility. pgpro_bindump offers the following benefits:

  • Simplified communication: pgpro_bindump does not require the SSH protocol and associated configurations to connect to a database server.

  • Integrated WAL transfer: WAL files are transferred together with data files, reducing the risk of data loss during backups.

  • Optimized storage: Files are not copied as they are, but are archived using the pg_probackup3 own format, CBOR (RFC 8949 Concise Binary Object Representation), which enhances the storage efficiency.

F.43.1. Architecture #

A dedicated backend process is created to handle connections from external applications. You can use the pg_probackup3 utility or a custom application to send requests to this backend.

Multiple connections can be established for the same application if needed.

Since the pgpro_bindump module has direct access to the database files, it does not require any additional permissions.

pgpro_bindump uses the libpgprobackup, along with its built-in encoder module, to send data to the application.

The walsender_plugin_libraries plugin has been added to simplify the process of adding and utilizing new replication commands, including custom ones.

F.43.2. Limitations and Requirements #

pgpro_bindump currently has the following limitations and requirements:

  • libpgprobackup.so and libpb3_encoder.so are required for pgpro_bindump operation.

F.43.3. Installation and Setup #

Once you complete Postgres Pro installation, follow the steps below.

  1. Specify the following parameters in the postgresql.conf file: 

    shared_preload_libraries = 'pgpro_bindump'
wal_level = 'replica' # or 'logical'
walsender_plugin_libraries = 'pgpro_bindump'

  2. Restart the Postgres Pro instance.

F.43.4. Replication Commands #

The pgpro_bindump module implements the following replication commands:

  • just-noop — a do-nothing command.

  • start_backup — starts the backup process.

  • copy_files — copies files in the CBOR format.

  • stop_backup — stops the backup process.

The backup process runs in the PRO mode — an advanced data source mode that incorporates all available optimizations and enhancements, including the proprietary replication protocol and specialized replication commands.

To send commands from a user application to pgpro_bindump and to receive the output data from pgpro_bindump by the application, use libpq.

Note

An established connection to a running Postgres Pro server is required.

F.43.4.1. start_backup #

start_backup LABEL label, INCR_MODE incr_mode, START_LSN start_lsn, TRANSFER_MODE transfer_mode,
[TRANSFER_BUFFER_SIZE buffer_size]

Starts the backup process. Sends the following data to the application:

  • start_lsn — LSN at the start of the backup.

  • start_tli — the timeline ID at the start of the backup.

  • min_xid — the minimal transaction ID at the start of the backup.

  • min_multixact — the minimal multixact ID at the start of the backup.

  • tablespace_list — a list of tablespaces included in the backup.

  • database_list — a list of databases included in the backup.

LABEL label

The name of the backup.

INCR_MODE incr_mode

The type of incremental backup. Possible values: PTRACK, DELTA.

START_LSN start_lsn

An XLogRecPtr value representing LSN at the start of the backup.

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

F.43.4.2. copy_files #

copy_files VERIFY_CHECKSUMS [TRUE | FALSE], COMPRESS_ALG compress_alg, COMPRESS_LVL compress_lvl, TRANSFER_MODE transfer_mode,
[INCR_MODE incr_mode], [START_LSN start_lsn], [TRANSFER_BUFFER_SIZE buffer_size],
[WORKER_INFO number_of_workers:worker_id]

Forwards blocks of data files and WAL files packaged in the CBOR format.

VERIFY_CHECKSUMS

A boolean value (true or false) indicating whether to verify checksums.

COMPRESS_ALG compress_alg

The compression algorithm to be used. Possible values: zlib, zstd, lz4, none.

COMPRESS_LVL compress_lvl

The compression level. Possible values: 1, 2, 3, 4, 5.

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

INCR_MODE incr_mode

The type of incremental backup. Possible values: PTRACK, DELTA.

START_LSN start_lsn

An XLogRecPtr value representing LSN at the start of the backup.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

WORKER_INFO number_of_workers:worker_id

The number of workers and the worker ID.

F.43.4.3. stop_backup #

stop_backup STREAM [TRUE | FALSE], COMPRESS_ALG compress_alg, COMPRESS_LVL compress_lvl,
[EXTERNALS externals] [TRANSFER_BUFFER_SIZE buffer_size], [TRANSFER_MODE transfer_mode]

Completes the copy process and closes the existing connection. Sends the following data to the application:

  • stop_lsn — LSN at the end of the backup.

  • stop_tli — the timeline ID at the end of the backup.

  • The backup.control file.

  • A set of WAL files (only if WAL streaming is enabled).

STREAM

A boolean value (true or false) indicating whether WAL streaming is enabled.

COMPRESS_ALG compress_alg

The compression algorithm to be used. Possible values: zlib, zstd, lz4, none.

COMPRESS_LVL compress_lvl

The compression level. Possible values: 1, 2, 3, 4, 5.

EXTERNALS externals

A list of external directories.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

F.43.5. Authors #

Postgres Professional, Moscow, Russia.

Prev Up Next
F.42. pg_proaudit — enables detailed logging of various security events Home F.44. pg_prewarm — preload relation data into buffer caches
pdf epub
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy