Updated polynomial selection

[10metreh]

Originally Posted by EdH

<snip>

Thanks for any advice. . . and for all the work you put into this!

Looks like msieve failed to find any polys and there was then an error because the script couldn't locate the poly file, which isn't surprising as it didn't exist. Does this also happen with this number on your other system?

[Batalov]

On x86_64, a dozen small jobs came and went without any problems since the switch to the new binary.

[EdH]

Originally Posted by 10metreh

Looks like msieve failed to find any polys and there was then an error because the script couldn't locate the poly file, which isn't surprising as it didn't exist. Does this also happen with this number on your other system?

No, it doesn't work on the other machine either. Basically, the c97 works on both and the c102 fails on both.

I watched more closely this time and found that it did only take about 1.5 minutes to return, but the system mentioned the failure to find a poly and then immediately started again. It failed the second try and died.

The c97 says it will run for .28 hour and seems correct. The c102 says .42 hour and returns in about 1.5 minutes. More later - I'm being called away. . .

[warut]

I did two more comparisons. The first was on C118 of 89222701^19-1.

From old version:

N 8837749134432835806195070022260085293230408134334557837785022990091369943335064126559771950700120538497543066264805577
SKEW 152586.60 
A5 7800
A4 -434442398
A3 374317654297266
A2 -51518077588729575083
A1 -9194728976614280772808512
A0 -31249598269629476722371712085
R1 3811599996679
R0 -64691868997542707118834
skew 152586.60, size 2.349e-011, alpha -6.725, combined = 3.440e-010 rroots = 3

From new version:

N 8837749134432835806195070022260085293230408134334557837785022990091369943335064126559771950700120538497543066264805577
SKEW 145029.30 
A5 8208
A4 -886026915
A3 -590168648007458
A2 17619088865183789852
A1 4801626438984607946945394
A0 -41185957167849307193226488313
R1 3846551399719
R0 -64035591815269946028524
skew 145029.30, size 2.499e-011, alpha -6.052, combined = 3.589e-010 rroots = 5

In this case, the new poly has higher score and did sieve faster.

The second test was on C119 of 1501489387^17-1, and the result is very interesting.

From old version:

N 49051250488250332964105132536210364412403538501460947746946260536246841041442844030066770905907500315930955943508134137
SKEW 232645.04 
A5 15060
A4 2625466784
A3 -848598148723481
A2 -226352786741367690238
A1 9703372046994915907276560
A0 4744332674855990753222837678400
R1 6176358528101
R0 -79903141015694543301389
skew 232645.04, size 2.700e-011, alpha -7.906, combined = 3.694e-010 rroots = 3

From new version:

N 49051250488250332964105132536210364412403538501460947746946260536246841041442844030066770905907500315930955943508134137
SKEW 258403.20 
A5 780
A4 394926913
A3 -159973478651993
A2 -18112496710022114576
A1 4699438625933033314884249
A0 237569645510482655941466299299
R1 2760211179763
R0 -144447575183681804011184
skew 258403.20, size 2.197e-011, alpha -5.372, combined = 3.369e-010 rroots = 3

In this case, the old poly has significantly higher score and did sieve faster. How did the new msieve miss this poly?

[jasonp]

The poly selection as implemented in msieve is not very deterministic, because the search space is so large. If an old version finds a good polynomial that says nothing about whether a newer version could find the same polynomial. Even the same code running on a different machine may not find the same polynomial, because there is a limit on wall-clock time that the code will spend on any given leading algebraic coefficient (and for large problems the code will randomly pick a small part of the search space and search only that). You can only hope that a given version will find some *other* polynomial that is hopefully as good or better than before.

[10metreh]

Originally Posted by EdH

No, it doesn't work on the other machine either. Basically, the c97 works on both and the c102 fails on both.

I watched more closely this time and found that it did only take about 1.5 minutes to return, but the system mentioned the failure to find a poly and then immediately started again. It failed the second try and died.

The c97 says it will run for .28 hour and seems correct. The c102 says .42 hour and returns in about 1.5 minutes. More later - I'm being called away. . .

I'm guessing this is the C102 from aliquot 121176 line 1201. I tried doing polynomial selection on that with the new CPU version (I don't have an nvidia GPU), and my run finished in 16 seconds! It did find two polys though (both with a leading coefficient of 1452), this being the better one:

# norm 9.608746e-010 alpha -4.936962 e 2.547e-009 rroots 1
skew: 21131.85
c0:  1600054119430816404941596
c1:  351741744142373229717
c2: -22930514643462185
c3: -259919144097
c4:  25321942
c5:  1452
Y0: -43428623886994629249
Y1:  32070619501

This looks like something to do with the degree 4/5 crossover being incorrectly compensated for.

[jrk]

The c102 says .42 hour and returns in about 1.5 minutes.

Looks like you're running out of A5. For c102, by default it stops at about A5=4000. If you want to search further, you can request a bigger range to test by specifying -np1 A5_min,A5_max on the command line for msieve. We may want to tweak the default range.

[fivemack]

I'm wondering if this might be an incompatibility / bug with gmp-5.0.1. Builds on my ubuntu machines against libgmp3-dev work fine.

Error came on

poly 0 77173 46567 55849 5953485316612104235062421
algebraic poly rootfinder failed

having built msieve against gmp-5.0.1-x86-64-i7-install/include (whose contents are what it says on the packet) and running

LD_LIBRARY_PATH=~/math/gmp-5.0.1-x86-64-i7-install/lib ~/math/msieve-svn-linux/trunk/msieve -v -np 93001,93999

with worktodo.ini containing

696005113603274146249087406461080151837868785491046290335993731206770181299351306725449365913668810065423694108036416675915143109

[EdH]

Originally Posted by 10metreh

I'm guessing this is the C102 from aliquot 121176 line 1201. I tried doing polynomial selection on that with the new CPU version (I don't have an nvidia GPU), and my run finished in 16 seconds! It did find two polys though (both with a leading coefficient of 1452), this being the better one:
<snip>
This looks like something to do with the degree 4/5 crossover being incorrectly compensated for.

This was from 125082:1988:

738677034112727630801795082875338426629435597713265653277611629383222178533167933102186322561385336223

1.47 worked and gnfs returned factors this morning:

[Dec 21 2010, 07:27:19] *** prp42 = 312737738894660776926271449719691462820837
[Dec 21 2010, 07:27:19] *** prp61 = 2361969606621526652678994236908166532945737170991242903747379

Originally Posted by jrk

Looks like you're running out of A5. For c102, by default it stops at about A5=4000. If you want to search further, you can request a bigger range to test by specifying -np1 A5_min,A5_max on the command line for msieve. We may want to tweak the default range.

Unfortunately, my intervention would mean modifying either Aliqueit or factmsieve.py, since I'm running this via Aliqueit. Perhaps I should, for a test run, anyway. If I get a chance, I may try that a little later today.

Thanks!

[EdH]

Bummer! It failed on one of my Ubuntu machines, too.

-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.    Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 1 Core with 1 hyper-thread per Core
-> Working with NAME = test
-> Error: Polynomial file test.poly does not exist!
-> Found n = 835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619.
-> Running polynomial selection ...
-> ./msieve -s ../../Aliqueit/ggnfs_835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619/test.dat -l ../../Aliqueit/ggnfs_835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619/test.log -i ../../Aliqueit/ggnfs_835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619/test.ini -nf ../../Aliqueit/ggnfs_835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619/test.fb -v -np


Msieve v. 1.48
Wed Dec 22 10:01:02 2010
random seeds: 16fcd29b 061338f5
factoring 835989648594235417164528517312284786947851669414128953643421282359992456221670733279993029863509619 (99 digits)
Msieve Error: return value -4. Terminating...

How can I run make to compile an earlier version in my svn copy? (I forgot to save the previous file.) I've tried to revert my svn copy, but it won't compile. . .

[EdH]

Update to prior post, since my edit window has expired:

I had to delete the whole directory and then check out an earlier SVN version from scratch, rather than simply reverting to an earlier one. My Ubuntu machine is now happy again.