IntegerExponent Equivalent in Pari-GP

[a1call]

Took me days to pinpoint my script's bug to Pari GP's rounding error:

With deafault precision I get:

? 100-99.99
%2 = 0.010000000000000000000000000000000000181

I would like to get 0.01 or 0.010000000000000000000000000000000000000
Any advice to resolve it will be greatly appreciated.

Thank you in advance.

[science_man_88]

Originally Posted by a1call

Took me days to pinpoint my script's bug to Pari GP's rounding error:

With deafault precision I get:

? 100-99.99
%2 = 0.010000000000000000000000000000000000181

I would like to get 0.01 or 0.010000000000000000000000000000000000000
Any advice to resolve it will be greatly appreciated.

Thank you in advance.

turn your result into a string at a higher precision using \p precision or localprec and other newer functions before calling it and then use a loop or other means to select the first precision under the precision gp gives you back ? just a thought.

[a1call]

Originally Posted by science_man_88

turn your result into a string at a higher precision using \p precision or localprec and other newer functions before calling it and then use a loop or other means to select the first precision under the precision gp gives you back ? just a thought.

Thank you science_man_88 for the suggestion. I am trying something similar, but i'm hoping for a more general solution whereby I don't have to keep track of the precision.
Wolfram Alpha shows a similar "mouse-over/hover" error, but then gives the correct answer once the enter is pressed.

[a1call]

inconsistent rounding up & down,

makes it much more difficult to deal with:

? 100-99.999
%2 = 0.00099999999999999999999999999999999994282

[science_man_88]

Originally Posted by a1call

inconsistent rounding up & down,

makes it much more difficult to deal with:

? 100-99.999
%2 = 0.00099999999999999999999999999999999994282

turn both into fractions and you get an exact fraction answer.

[a1call]

Originally Posted by science_man_88

turn both into fractions and you get an exact fraction answer.

I just tried that by multiplying the sum by the fraction numerator, denominator function such that the sum is integers only then devide by denominator. It doesn't seem to work.

What works is truncating the lower digits by rounding, but i have to keep track of significant digits which is inconvenient (figuring out extra code).

[science_man_88]

Originally Posted by a1call

I just tried that by multiplying the sum by the fraction numerator, denominator function such that the sum is integers only then devide by denominator. It doesn't seem to work.

What works is truncating the lower digits by rounding, but i have to keep track of significant digits which is inconvenient (figuring out extra code).

try 10000/100 -9999/100 it will work then.

[chalsall]

Originally Posted by a1call

I just tried that by multiplying the sum by the fraction numerator, denominator function such that the sum is integers only then devide by denominator. It doesn't seem to work.

You're kidding, right?

You don't understand how floating point numbers are represented on digital systems?

[Dubslow]

a1call, you are definitely in need of reading (e.g.) this page: https://docs.python.org/3.5/tutorial/floatingpoint.html (and possibly sm88 too?)

[a1call]

Originally Posted by chalsall

You're kidding, right?

You don't understand how floating point numbers are represented on digital systems?

I think I do.

I essentially did this:

100-99.99 =100/100(100-99.99) = (10000-9999)/100

The error seems to be related to the Pari-GP keeping track of the number as much as possible in a/b form and only expanding to decimal point form when needed such as in sums.

[chalsall]

Originally Posted by a1call

The error seems to be related to the Pari-GP keeping track of the number as much as possible in a/b form and only expanding to decimal point form when needed such as in sums.

Trust the programmer to blame the language.