fix printf precision specifier for hex floats on non-ld80 archs
the code to perform rounding to the desired precision wrongly assumed the long double mantissa was an integral number of nibbles (hex digits) in length. this is true for 80-bit extended precision (64-bit mantissa) but not for double (53) or quad (113). scale the rounding value by 1<<(LDBL_MANT_DIG%4) to compensate.
This commit is contained in:
@@ -220,6 +220,7 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
|
||||
else re=LDBL_MANT_DIG/4-1-p;
|
||||
|
||||
if (re) {
|
||||
round *= 1<<(LDBL_MANT_DIG%4);
|
||||
while (re--) round*=16;
|
||||
if (*prefix=='-') {
|
||||
y=-y;
|
||||
|
||||
Reference in New Issue
Block a user