Fix failure of btree_gin indexscans with "char" type and </<= operators.

tglsfdc · tglsfdc · commit a4957b5a72dc · 2021-08-10T18:10:30.000-04:00
As a result of confusion about whether the "char" type is signed or unsigned, scans for index searches like "col < 'x'" or "col <= 'x'" would start at the middle of the index not the left end, thus missing many or all of the entries they should find. Fortunately, this is not a symptom of index corruption. It's only the search logic that is broken, and we can fix it without unpleasant side-effects. Per report from Jason Kim. This has been wrong since btree_gin's beginning, so back-patch to all supported branches. Discussion: https://postgr.es/m/20210810001649.htnltbh7c63re42p@jasonk.me
diff --git a/contrib/btree_gin/btree_gin.c b/contrib/btree_gin/btree_gin.c
@@ -357,7 +357,7 @@ GIN_SUPPORT(bpchar, true, leftmostvalue_text, bpcharcmp)
 static Datum
 leftmostvalue_char(void)
 {
-	return CharGetDatum(SCHAR_MIN);
+	return CharGetDatum(0);
 }
 
 GIN_SUPPORT(char, false, leftmostvalue_char, btcharcmp)
diff --git a/contrib/btree_gin/expected/char.out b/contrib/btree_gin/expected/char.out
@@ -7,12 +7,19 @@ CREATE INDEX idx_char ON test_char USING gin (i);
 SELECT * FROM test_char WHERE i<'d'::"char" ORDER BY i;
  i 
 ---
-(0 rows)
+ a
+ b
+ c
+(3 rows)
 
 SELECT * FROM test_char WHERE i<='d'::"char" ORDER BY i;
  i 
 ---
-(0 rows)
+ a
+ b
+ c
+ d
+(4 rows)
 
 SELECT * FROM test_char WHERE i='d'::"char" ORDER BY i;
  i 

Original file line number	Diff line number	Diff line change
`@@ -357,7 +357,7 @@ GIN_SUPPORT(bpchar, true, leftmostvalue_text, bpcharcmp)`
`357`	`357`	`static Datum`
`358`	`358`	`leftmostvalue_char(void)`
`359`	`359`	`{`
`360`		`- return CharGetDatum(SCHAR_MIN);`
	`360`	`+ return CharGetDatum(0);`
`361`	`361`	`}`
`362`	`362`
`363`	`363`	`GIN_SUPPORT(char, false, leftmostvalue_char, btcharcmp)`