Skip to content

Commit 775f1b3

Browse files
committed
Provide for parallel restoration from a custom format archive. Each data and
post-data step is run in a separate worker child (a thread on Windows, a child process elsewhere) up to the concurrent number specified by the new pg_restore command-line --multi-thread | -m switch. Andrew Dunstan, with some editing by Tom Lane.
1 parent 3a5b773 commit 775f1b3

File tree

11 files changed

+1509
-276
lines changed

11 files changed

+1509
-276
lines changed

doc/src/sgml/ref/pg_restore.sgml

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<!-- $PostgreSQL: pgsql/doc/src/sgml/ref/pg_restore.sgml,v 1.77 2009/01/05 16:54:36 tgl Exp $ -->
1+
<!-- $PostgreSQL: pgsql/doc/src/sgml/ref/pg_restore.sgml,v 1.78 2009/02/02 20:07:36 adunstan Exp $ -->
22

33
<refentry id="APP-PGRESTORE">
44
<refmeta>
@@ -241,6 +241,28 @@
241241
</listitem>
242242
</varlistentry>
243243

244+
<varlistentry>
245+
<term><option>-m <replaceable class="parameter">number-of-threads</replaceable></option></term>
246+
<term><option>--multi-thread=<replaceable class="parameter">number-of-threads</replaceable></option></term>
247+
<listitem>
248+
<para>
249+
Run the most time-consuming parts of <application>pg_restore</>
250+
&mdash; those which load data, create indexes, or create
251+
constraints &mdash; using multiple concurrent connections to the
252+
database. This option can dramatically reduce the time to restore a
253+
large database to a server running on a multi-processor machine.
254+
</para>
255+
256+
<para>
257+
This option is ignored when emitting a script rather than connecting
258+
directly to a database server. Multiple threads cannot be used
259+
together with <option>--single-transaction</option>. Also, the input
260+
must be a plain file (not, for example, a pipe), and at present only
261+
the custom archive format is supported.
262+
</para>
263+
</listitem>
264+
</varlistentry>
265+
244266
<varlistentry>
245267
<term><option>-n <replaceable class="parameter">namespace</replaceable></option></term>
246268
<term><option>--schema=<replaceable class="parameter">schema</replaceable></option></term>

src/bin/pg_dump/pg_backup.h

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
*
1616
*
1717
* IDENTIFICATION
18-
* $PostgreSQL: pgsql/src/bin/pg_dump/pg_backup.h,v 1.48 2009/01/05 16:54:36 tgl Exp $
18+
* $PostgreSQL: pgsql/src/bin/pg_dump/pg_backup.h,v 1.49 2009/02/02 20:07:36 adunstan Exp $
1919
*
2020
*-------------------------------------------------------------------------
2121
*/
@@ -53,6 +53,14 @@ typedef enum _archiveMode
5353
archModeRead
5454
} ArchiveMode;
5555

56+
typedef enum _teSection
57+
{
58+
SECTION_NONE = 1, /* COMMENTs, ACLs, etc; can be anywhere */
59+
SECTION_PRE_DATA, /* stuff to be processed before data */
60+
SECTION_DATA, /* TABLE DATA, BLOBS, BLOB COMMENTS */
61+
SECTION_POST_DATA /* stuff to be processed after data */
62+
} teSection;
63+
5664
/*
5765
* We may want to have some more user-readable data, but in the mean
5866
* time this gives us some abstraction and type checking.
@@ -124,6 +132,7 @@ typedef struct _restoreOptions
124132
int suppressDumpWarnings; /* Suppress output of WARNING entries
125133
* to stderr */
126134
bool single_txn;
135+
int number_of_threads;
127136

128137
bool *idWanted; /* array showing which dump IDs to emit */
129138
} RestoreOptions;
@@ -152,7 +161,8 @@ extern void ArchiveEntry(Archive *AHX,
152161
const char *tag,
153162
const char *namespace, const char *tablespace,
154163
const char *owner, bool withOids,
155-
const char *desc, const char *defn,
164+
const char *desc, teSection section,
165+
const char *defn,
156166
const char *dropStmt, const char *copyStmt,
157167
const DumpId *deps, int nDeps,
158168
DataDumperPtr dumpFn, void *dumpArg);

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy