Avoid integer overflow while sifting-up a heap in tuplesort.c.

tglsfdc · tglsfdc · commit 09c598898166 · 2017-07-12T13:24:16.000-04:00
If the number of tuples in the heap exceeds approximately INT_MAX/2, this loop's calculation "2*i+1" could overflow, resulting in a crash. Fix it by using unsigned int rather than int for the relevant local variables; that shouldn't cost anything extra on any popular hardware. Per bug #14722 from Sergey Koposov. Original patch by Sergey Koposov, modified by me per a suggestion from Heikki Linnakangas to use unsigned int not int64. Back-patch to 9.4, where tuplesort.c grew the ability to sort as many as INT_MAX tuples in-memory (commit 263865a). Discussion: https://postgr.es/m/20170629161637.1478.93109@wrigleys.postgresql.org
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
@@ -3800,7 +3800,7 @@ tuplesort_heap_siftup(Tuplesortstate *state, bool checkIndex)
 {
 	SortTuple  *memtuples = state->memtuples;
 	SortTuple  *tuple;
-	int			i,
+	unsigned int i,
 				n;
 
 	Assert(!checkIndex || state->currentRun == RUN_FIRST);
@@ -3809,12 +3809,17 @@ tuplesort_heap_siftup(Tuplesortstate *state, bool checkIndex)
 
 	CHECK_FOR_INTERRUPTS();
 
+	/*
+	 * state->memtupcount is "int", but we use "unsigned int" for i, j, n.
+	 * This prevents overflow in the "2 * i + 1" calculation, since at the top
+	 * of the loop we must have i < n <= INT_MAX <= UINT_MAX/2.
+	 */
 	n = state->memtupcount;
 	tuple = &memtuples[n];		/* tuple that must be reinserted */
 	i = 0;						/* i is where the "hole" is */
 	for (;;)
 	{
-		int			j = 2 * i + 1;
+		unsigned int j = 2 * i + 1;
 
 		if (j >= n)
 			break;
