Commit 7d9a473

committed

Improve type numeric's calculations for ln(), log(), exp(), pow().

Set the "rscales" for intermediate-result calculations to ensure that suitable numbers of significant digits are maintained throughout. The previous coding hadn't thought this through in any detail, and as a result could deliver results with many inaccurate digits, or in the worst cases even fail with divide-by-zero errors as a result of losing all nonzero digits of intermediate results. In exp_var(), get rid entirely of the logic that separated the calculation into integer and fractional parts: that was neither accurate nor particularly fast. The existing range-reduction method of dividing by 2^n can be applied across the full input range instead of only 0..1, as long as we are careful to set an appropriate rscale for each step. Also fix the logic in mul_var() for shortening the calculation when the caller asks for fewer output digits than an exact calculation would require. This bug doesn't affect simple multiplications since that code path asks for an exact result, but it does contribute to accuracy issues in the transcendental math functions. In passing, improve performance of mul_var() a bit by forcing the shorter input to be on the left, thus reducing the number of iterations of the outer loop and probably also reducing the number of carry-propagation steps needed. This is arguably a bug fix, but in view of the lack of field complaints, it does not seem worth the risk of back-patching. Dean Rasheed

1 parent e57646e commit 7d9a473Copy full SHA for 7d9a473

5 files changed

+2853

-227

lines changed

src
- backend/utils/adt
  - numeric.c
- test/regress
  - expected
    - numeric.out
    - numeric_big.out
  - sql
    - numeric.sql
    - numeric_big.sql

5 files changed

+2853

-227

lines changed

Comments

(0)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Commit 7d9a473

5 files changed

5 files changed

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!

File tree

5 files changed

5 files changed

0 commit comments

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!