mersenneforum.org Choosing Between Multiple Poly Files
 Register FAQ Search Today's Posts Mark Forums Read

 2018-03-10, 23:38 #1 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 31×109 Posts Choosing Between Multiple Poly Files I'm trying to run msieve poly select on several machines at once with a short deadline and then programatically choose the best. I can run separate machines on separate ranges with poly_deadline=300, or such, but the .fb from each machine has no score data. I can harvest the score data from all the log files, but is there a better way? I tried breaking the tasks into the separate selection stages, but that added way too much time. Thanks
 2018-03-10, 23:59 #2 Dubslow Basketry That Evening!     "Bunslow the Bold" Jun 2011 40
2018-03-11, 00:25   #3
swellman

Jun 2012

1011001110002 Posts

There’s this, hope it helps.

Quote:
 Originally Posted by fivemack Just put the polynomial in msieve format into an msieve.fb file and run msieve -nc1 with no msieve.dat file present Code: skew 28549120.00, size 8.396e-18, alpha -6.740, combined = 6.137e-14 rroots = 3
There’s also the log file, it has an output line which includes the highest combined e-score.

Last fiddled with by swellman on 2018-03-11 at 00:32

2018-03-11, 16:19   #4
EdH

"Ed Hall"
Dec 2009

64638 Posts

Quote:
 Originally Posted by Dubslow
This is of great interest, but I can't seem to make the function stand alone without a bit more learnin' - drat!
Quote:
 Originally Posted by swellman There’s this, hope it helps. ... There’s also the log file, it has an output line which includes the highest combined e-score.
I guess I'm too ignorant to get the msieve/.fb method to work. All I get are errors or missing file notices. Is there an example of this somewhere?

I'd already considered the log file and may just have to go that route, since it's something I can do.

Thanks guys!

Last fiddled with by EdH on 2018-03-11 at 16:19

 2018-03-11, 16:47 #5 chris2be8     Sep 2009 1,913 Posts Try this: Code: #!/usr/bin/perl # Code from mkpoly.pl to find the best polynomial in NAME.*.msieve.dat.p after -npr has been split over several CPUs. use warnings; $bestscore = 0;$NAME=$ARGV[0]; open(IF, "$NAME.n") or die("Cannot read $NAME.n$!"); @numberinf=; close(IF) or warn "Problem closing $NAME.n$!"; #print "$NAME.n contains:\n",@numberinf; # DEBUG foreach my$file (glob "$NAME.*.p") { open(IN,"<$file") or die "Can't read $file$!"; print "Reading $file\n"; # Debug my$wantit = 0; # Do we want this line? while() { my @word = split; # Input is like: # norm 1.498232e-10 alpha -5.818790 e 9.344e-10 rroots 3 # 1 2 3 4 5 6 7 8 if ($word[1] eq "norm") {$wantit = 0; # Assume we don't want this poly if ($bestscore <$word[6]) { $bestscore =$word[6]; print "Best score so far came from $file:$_"; $wantit = 1; @poly = (); # Forget last polynomial } } push @poly,$_ if($wantit); #print$_ if($wantit); # debugging } close(IN) or warn "Problem closing$file $!"; } if ($bestscore == 0 ) { die "Could not find any polynomials"; } # Now build $NAME.poly open(OUT,">$NAME.poly") or die "Can't open $NAME.poly$!"; chomp $numberinf[-1]; # The last line in$NAME.n may or may not end with a newline. # So make sure it doesn't now, then always add one to $NAME.poly print OUT @numberinf,"\n"; # Lines read from$NAME.n print OUT @poly; # The best polynomial print OUT "type: gnfs\n"; # And say it's GNFS close(OUT) or warn "Problem closing $NAME.poly$!"; It expects 1 parameter, the prefix for the files. Put the prefix.n file and all the prefix.*.p files in 1 dir, then run it. Chris
2018-03-11, 22:53   #6
EdH

"Ed Hall"
Dec 2009

31×109 Posts

Quote:
 Originally Posted by chris2be8 Try this: Code: #!/usr/bin/perl # Code from mkpoly.pl to find the best polynomial in NAME.*.msieve.dat.p after -npr has been split over several CPUs. use warnings; $bestscore = 0;$NAME=$ARGV[0]; open(IF, "$NAME.n") or die("Cannot read $NAME.n$!"); @numberinf=; close(IF) or warn "Problem closing $NAME.n$!"; #print "$NAME.n contains:\n",@numberinf; # DEBUG foreach my$file (glob "$NAME.*.p") { open(IN,"<$file") or die "Can't read $file$!"; print "Reading $file\n"; # Debug my$wantit = 0; # Do we want this line? while() { my @word = split; # Input is like: # norm 1.498232e-10 alpha -5.818790 e 9.344e-10 rroots 3 # 1 2 3 4 5 6 7 8 if ($word[1] eq "norm") {$wantit = 0; # Assume we don't want this poly if ($bestscore <$word[6]) { $bestscore =$word[6]; print "Best score so far came from $file:$_"; $wantit = 1; @poly = (); # Forget last polynomial } } push @poly,$_ if($wantit); #print$_ if($wantit); # debugging } close(IN) or warn "Problem closing$file $!"; } if ($bestscore == 0 ) { die "Could not find any polynomials"; } # Now build $NAME.poly open(OUT,">$NAME.poly") or die "Can't open $NAME.poly$!"; chomp $numberinf[-1]; # The last line in$NAME.n may or may not end with a newline. # So make sure it doesn't now, then always add one to $NAME.poly print OUT @numberinf,"\n"; # Lines read from$NAME.n print OUT @poly; # The best polynomial print OUT "type: gnfs\n"; # And say it's GNFS close(OUT) or warn "Problem closing $NAME.poly$!"; It expects 1 parameter, the prefix for the files. Put the prefix.n file and all the prefix.*.p files in 1 dir, then run it. Chris
Perhaps I am just not meant to be able to do this:
Code:
$perl polySel.pl rsa130 syntax error at polySel.pl line 12, near "=;" BEGIN not safe after errors--compilation aborted at polySel.pl line 16.  2018-03-12, 03:01 #7 EdH "Ed Hall" Dec 2009 Adirondack Mtns 337910 Posts I changed: Code: @numberinf=; to: Code: @numberinf=; and it seems to read the rsa130.n file, but now I get: Code: ... Use of uninitialized value$_ in split at polySel.pl line 21. Use of uninitialized value $word[1] in string eq at polySel.pl line 25. Use of uninitialized value$_ in split at polySel.pl line 21. Use of uninitialized value $word[1] in string eq at polySel.pl line 25. Use of uninitialized value$_ in split at polySel.pl line 21. Use of uninitialized value \$word[1] in string eq at polySel.pl line 25. ... I can't see where @word gets filled...
2018-03-12, 16:40   #8
chris2be8

Sep 2009

1,913 Posts

Hello,

Sorry, html and perl don't get on very well together, bits html thinks are an unknown tag disappear. Line 20 should read:

while(<IN>) {

or try the attached version.

Chris
Attached Files
 refindpoly.pl.txt (1.6 KB, 73 views)

2018-03-12, 18:02   #9
xilman
Bamboozled!

"𒉺𒌌𒇷𒆷𒀭"
May 2003
Down not across

34×53 Posts

Quote:
 Originally Posted by EdH I'd already considered the log file and may just have to go that route, since it's something I can do.
It's what I do.

2018-03-12, 20:41   #10
EdH

"Ed Hall"
Dec 2009

31×109 Posts

Quote:
 Originally Posted by chris2be8 Hello, Sorry, html and perl don't get on very well together, bits html thinks are an unknown tag disappear. Line 20 should read: while() { or try the attached version. Chris
Ahh, Excellent! This is working quite well. Thank you!
Quote:
 Originally Posted by xilman It's what I do.
I hate to reinvent in a sub-par manner, which is what I would have done in this case. I was intending to find the best poly based on log info and then use the .fb file corresponding to the best poly, which would still leave factmsieve.py to convert the .fb to .poly. The perl script from Chris can take my combined .p file and give me the .poly. That's even better than I was planning.

Thanks for all the help, everyone...

 2018-03-15, 03:16 #11 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 31×109 Posts Thank you to everyone that helped me through this experiment. I was able to use my scripts to run msieve/gnfs via factmsieve.py across 32 machines of very diverse antiquity and near current hardware to factor RSA140 in just under 6 (5:42) hours. Not an exceptionally great achievement for others, but a pretty darn good one for me. Some notables: Of the 32 machines, 2 of them tested faster at sieving using b2's 64-bit asm sievers, while 30 of them ran slightly faster with machine specific compilations. Of note, the differences in speed were actually minuscule, on the order of ~0.006 s/rel. Although factmsieve.py suggests only using 5 to 10 machines for distributed factoring, I had 32 running in this test and 30 in a previous test and 2/3 in other tests. All seemed to run well in these tests, except: --with two machines, there was a considerable waste of sieving, in that the first check had 82% of the needed relations and the second check found 162%. This can be altered, depending on the size of the composite, within factmsieve.py or in the appropriate def file in ggnfs/bin. However, this modification would need to be done in all the copies on all the machines. --with many machines of various capability, a lot of them continue sieving when the host has already started Linear Algebra, only ceasing when they report their latest batch of (unneeded) relations. I hope to eventually document how I set everything up for these factoring runs in my How I ... blogorrhea posts. Thanks again everyone for the help.

 Similar Threads Thread Thread Starter Forum Replies Last Post VBCurtis Msieve 0 2016-04-11 21:33 wombatman Msieve 123 2013-08-27 11:27 Flatlander GPU Computing 4 2011-01-26 08:15 jasong Software 18 2005-12-30 20:23 BillW Software 1 2003-01-21 20:11

All times are UTC. The time now is 06:21.

Tue Oct 27 06:21:47 UTC 2020 up 47 days, 3:32, 0 users, load averages: 2.09, 2.77, 3.67