Quick fix for stack buffer overrun in rev.1.13. Oops. The prec == 1

arg to __kernel_rem_pio2() gives 53-bit (double) precision, not single
precision and/or the array dimension like I thought.  prec == 2 is
used in e_rem_pio2.c for double precision although it is documented
to be for 64-bit (extended) precision, and I just reduced it by 1
thinking that this would give the value suitable for 24-bit (float)
precision.  Reducing it 1 more to the documented value for float
precision doesn't actually work (it gives errors of ~0.75 ulps in the
reduced arg, but errors of much less than 0.5 ulps are needed; the bug
seems to be in kernel_rem_pio2.c).  Keep using a value 1 larger than
the documented value but supply an array large enough hold the extra
unused result from this.

The bug can also be fixed quickly by increasing init_jk[0] in
k_rem_pio2.c from 2 to 3.  This gives behaviour identical to using
prec == 1 except it doesn't create the extra result.  It isn't clear
how the precision bug affects higher precisions.  113-bit (quad) is
the largest precision, so there is no way to use a large precision
to fix it.

0 files changed, 0 insertions, 0 deletions