Add stack-overflow guards in set-operation planning.

tglsfdc · tglsfdc · commit e194f13838a5 · 2018-01-28T13:39:07.000-05:00
create_plan_recurse lacked any stack depth check. This is not per our normal coding rules, but I'd supposed it was safe because earlier planner processing is more complex and presumably should eat more stack. But bug #15033 from Andrew Grossman shows this isn't true, at least not for queries having the form of a many-thousand-way INTERSECT stack. Further testing showed that recurse_set_operations is also capable of being crashed in this way, since it likewise will recurse to the bottom of a parsetree before calling any support functions that might themselves contain any stack checks. However, its stack consumption is only perhaps a third of create_plan_recurse's. It's possible that this particular problem with create_plan_recurse can only manifest in 9.6 and later, since before that we didn't build a Path tree for set operations. But having seen this example, I now have no faith in the proposition that create_plan_recurse doesn't need a stack check, so back-patch to all supported branches. Discussion: https://postgr.es/m/20180127050845.28812.58244@wrigleys.postgresql.org
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
@@ -230,6 +230,9 @@ create_plan_recurse(PlannerInfo *root, Path *best_path)
 {
 	Plan	   *plan;
 
+	/* Guard against stack overflow due to overly complex plans */
+	check_stack_depth();
+
 	switch (best_path->pathtype)
 	{
 		case T_SeqScan:
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
@@ -220,6 +220,9 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
 					   int flag, List *refnames_tlist,
 					   List **sortClauses, double *pNumGroups)
 {
+	/* Guard against stack overflow due to overly complex setop nests */
+	check_stack_depth();
+
 	if (IsA(setOp, RangeTblRef))
 	{
 		RangeTblRef *rtr = (RangeTblRef *) setOp;

Original file line number	Diff line number	Diff line change
`@@ -230,6 +230,9 @@ create_plan_recurse(PlannerInfo root, Path best_path)`
`230`	`230`	`{`
`231`	`231`	`Plan *plan;`
`232`	`232`
	`233`	`+ /* Guard against stack overflow due to overly complex plans */`
	`234`	`+ check_stack_depth();`
	`235`	`+`
`233`	`236`	`switch (best_path->pathtype)`
`234`	`237`	`{`
`235`	`238`	`case T_SeqScan:`
Original file line number	Diff line number	Diff line change
`@@ -220,6 +220,9 @@ recurse_set_operations(Node setOp, PlannerInfo root,`
`220`	`220`	`int flag, List *refnames_tlist,`
`221`	`221`	`List *sortClauses, double pNumGroups)`
`222`	`222`	`{`
	`223`	`+ /* Guard against stack overflow due to overly complex setop nests */`
	`224`	`+ check_stack_depth();`
	`225`	`+`
`223`	`226`	`if (IsA(setOp, RangeTblRef))`
`224`	`227`	`{`
`225`	`228`	`RangeTblRef rtr = (RangeTblRef ) setOp;`