Skip to content

Commit fad153e

Browse files
committed
Rewrite the sinval messaging mechanism to reduce contention and avoid
unnecessary cache resets. The major changes are: * When the queue overflows, we only issue a cache reset to the specific backend or backends that still haven't read the oldest message, rather than resetting everyone as in the original coding. * When we observe backend(s) falling well behind, we signal SIGUSR1 to only one backend, the one that is furthest behind and doesn't already have a signal outstanding for it. When it finishes catching up, it will in turn signal SIGUSR1 to the next-furthest-back guy, if there is one that is far enough behind to justify a signal. The PMSIGNAL_WAKEN_CHILDREN mechanism is removed. * We don't attempt to clean out dead messages after every message-receipt operation; rather, we do it on the insertion side, and only when the queue fullness passes certain thresholds. * Split SInvalLock into SInvalReadLock and SInvalWriteLock so that readers don't block writers nor vice versa (except during the infrequent queue cleanout operations). * Transfer multiple sinval messages for each acquisition of a read or write lock.
1 parent 30dc388 commit fad153e

File tree

8 files changed

+413
-244
lines changed

8 files changed

+413
-244
lines changed

src/backend/postmaster/postmaster.c

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
*
3838
*
3939
* IDENTIFICATION
40-
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.558 2008/06/06 22:35:22 alvherre Exp $
40+
* $PostgreSQL: pgsql/src/backend/postmaster/postmaster.c,v 1.559 2008/06/19 21:32:56 tgl Exp $
4141
*
4242
* NOTES
4343
*
@@ -3829,16 +3829,6 @@ sigusr1_handler(SIGNAL_ARGS)
38293829
load_role();
38303830
}
38313831

3832-
if (CheckPostmasterSignal(PMSIGNAL_WAKEN_CHILDREN))
3833-
{
3834-
/*
3835-
* Send SIGUSR1 to all children (triggers CatchupInterruptHandler).
3836-
* See storage/ipc/sinval[adt].c for the use of this.
3837-
*/
3838-
if (Shutdown <= SmartShutdown)
3839-
SignalChildren(SIGUSR1);
3840-
}
3841-
38423832
if (CheckPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER) &&
38433833
PgArchPID != 0)
38443834
{

src/backend/storage/ipc/sinval.c

Lines changed: 69 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
*
99
*
1010
* IDENTIFICATION
11-
* $PostgreSQL: pgsql/src/backend/storage/ipc/sinval.c,v 1.85 2008/03/17 11:50:26 alvherre Exp $
11+
* $PostgreSQL: pgsql/src/backend/storage/ipc/sinval.c,v 1.86 2008/06/19 21:32:56 tgl Exp $
1212
*
1313
*-------------------------------------------------------------------------
1414
*/
@@ -17,19 +17,17 @@
1717
#include "access/xact.h"
1818
#include "commands/async.h"
1919
#include "miscadmin.h"
20-
#include "storage/backendid.h"
2120
#include "storage/ipc.h"
22-
#include "storage/proc.h"
2321
#include "storage/sinvaladt.h"
2422
#include "utils/inval.h"
2523

2624

2725
/*
2826
* Because backends sitting idle will not be reading sinval events, we
2927
* need a way to give an idle backend a swift kick in the rear and make
30-
* it catch up before the sinval queue overflows and forces everyone
31-
* through a cache reset exercise. This is done by broadcasting SIGUSR1
32-
* to all backends when the queue is threatening to become full.
28+
* it catch up before the sinval queue overflows and forces it to go
29+
* through a cache reset exercise. This is done by sending SIGUSR1
30+
* to any backend that gets too far behind.
3331
*
3432
* State for catchup events consists of two flags: one saying whether
3533
* the signal handler is currently allowed to call ProcessCatchupEvent
@@ -47,67 +45,101 @@ static void ProcessCatchupEvent(void);
4745

4846

4947
/*
50-
* SendSharedInvalidMessage
51-
* Add a shared-cache-invalidation message to the global SI message queue.
48+
* SendSharedInvalidMessages
49+
* Add shared-cache-invalidation message(s) to the global SI message queue.
5250
*/
5351
void
54-
SendSharedInvalidMessage(SharedInvalidationMessage *msg)
52+
SendSharedInvalidMessages(const SharedInvalidationMessage *msgs, int n)
5553
{
56-
bool insertOK;
57-
58-
insertOK = SIInsertDataEntry(msg);
59-
if (!insertOK)
60-
elog(DEBUG4, "SI buffer overflow");
54+
SIInsertDataEntries(msgs, n);
6155
}
6256

6357
/*
6458
* ReceiveSharedInvalidMessages
6559
* Process shared-cache-invalidation messages waiting for this backend
6660
*
61+
* We guarantee to process all messages that had been queued before the
62+
* routine was entered. It is of course possible for more messages to get
63+
* queued right after our last SIGetDataEntries call.
64+
*
6765
* NOTE: it is entirely possible for this routine to be invoked recursively
6866
* as a consequence of processing inside the invalFunction or resetFunction.
69-
* Hence, we must be holding no SI resources when we call them. The only
70-
* bad side-effect is that SIDelExpiredDataEntries might be called extra
71-
* times on the way out of a nested call.
67+
* Furthermore, such a recursive call must guarantee that all outstanding
68+
* inval messages have been processed before it exits. This is the reason
69+
* for the strange-looking choice to use a statically allocated buffer array
70+
* and counters; it's so that a recursive call can process messages already
71+
* sucked out of sinvaladt.c.
7272
*/
7373
void
7474
ReceiveSharedInvalidMessages(
7575
void (*invalFunction) (SharedInvalidationMessage *msg),
7676
void (*resetFunction) (void))
7777
{
78-
SharedInvalidationMessage data;
79-
int getResult;
80-
bool gotMessage = false;
78+
#define MAXINVALMSGS 32
79+
static SharedInvalidationMessage messages[MAXINVALMSGS];
80+
/*
81+
* We use volatile here to prevent bugs if a compiler doesn't realize
82+
* that recursion is a possibility ...
83+
*/
84+
static volatile int nextmsg = 0;
85+
static volatile int nummsgs = 0;
8186

82-
for (;;)
87+
/* Deal with any messages still pending from an outer recursion */
88+
while (nextmsg < nummsgs)
8389
{
84-
/*
85-
* We can discard any pending catchup event, since we will not exit
86-
* this loop until we're fully caught up.
87-
*/
88-
catchupInterruptOccurred = 0;
90+
SharedInvalidationMessage *msg = &messages[nextmsg++];
8991

90-
getResult = SIGetDataEntry(MyBackendId, &data);
92+
invalFunction(msg);
93+
}
94+
95+
do
96+
{
97+
int getResult;
98+
99+
nextmsg = nummsgs = 0;
100+
101+
/* Try to get some more messages */
102+
getResult = SIGetDataEntries(messages, MAXINVALMSGS);
91103

92-
if (getResult == 0)
93-
break; /* nothing more to do */
94104
if (getResult < 0)
95105
{
96106
/* got a reset message */
97107
elog(DEBUG4, "cache state reset");
98108
resetFunction();
109+
break; /* nothing more to do */
99110
}
100-
else
111+
112+
/* Process them, being wary that a recursive call might eat some */
113+
nextmsg = 0;
114+
nummsgs = getResult;
115+
116+
while (nextmsg < nummsgs)
101117
{
102-
/* got a normal data message */
103-
invalFunction(&data);
118+
SharedInvalidationMessage *msg = &messages[nextmsg++];
119+
120+
invalFunction(msg);
104121
}
105-
gotMessage = true;
106-
}
107122

108-
/* If we got any messages, try to release dead messages */
109-
if (gotMessage)
110-
SIDelExpiredDataEntries(false);
123+
/*
124+
* We only need to loop if the last SIGetDataEntries call (which
125+
* might have been within a recursive call) returned a full buffer.
126+
*/
127+
} while (nummsgs == MAXINVALMSGS);
128+
129+
/*
130+
* We are now caught up. If we received a catchup signal, reset that
131+
* flag, and call SICleanupQueue(). This is not so much because we
132+
* need to flush dead messages right now, as that we want to pass on
133+
* the catchup signal to the next slowest backend. "Daisy chaining" the
134+
* catchup signal this way avoids creating spikes in system load for
135+
* what should be just a background maintenance activity.
136+
*/
137+
if (catchupInterruptOccurred)
138+
{
139+
catchupInterruptOccurred = 0;
140+
elog(DEBUG4, "sinval catchup complete, cleaning queue");
141+
SICleanupQueue(false, 0);
142+
}
111143
}
112144

113145

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy