mersenneforum.org - View Single Post

Prime95 · 2004-07-31, 17:34

Further study of this naughty case reveals no problem in the FFT code. Unfortunately, the FFT data number being squared does not exhibit "random" properties that generates low round-off errors.

I'm talking to the PFGW folks about possible workarounds. They have seen this many times in the past. Since prp3 does the last 50 squarings with round-off checking enabled, then I think it is safe to use. If you get this error in the last few iterations, I'll bet you have a probable prime and the next rev of prp3 will backtrack and handle properly.

2004-07-31, 17:34	#11
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2×59×67 Posts	Further study of this naughty case reveals no problem in the FFT code. Unfortunately, the FFT data number being squared does not exhibit "random" properties that generates low round-off errors. I'm talking to the PFGW folks about possible workarounds. They have seen this many times in the past. Since prp3 does the last 50 squarings with round-off checking enabled, then I think it is safe to use. If you get this error in the last few iterations, I'll bet you have a probable prime and the next rev of prp3 will backtrack and handle properly.