Msieve v1.49 feedback

Andi47 · 2011-06-18, 09:51

Quote:

Originally Posted by jasonp

Until then it's going to be all NFS polynomial selection, all the time.

hmmmm... what about multithreaded NFS poly selection (msieve -np -t <threads>)? one can do this with the new yafu nfs() functionality or by hand, but both ways are not suitable for fully automated aliquot sequence runs with aliqueit.

jasonp · 2011-06-18, 10:56

For at least the next few months, I'm only going to have the time to do some deep explorations on running stage 2 for very large problems. After that, I've been thinking of an overhaul of the code to use some of the nice stuff in YAFU.

chris2be8 · 2011-06-18, 16:41

Quote:

Originally Posted by Andi47

hmmmm... what about multithreaded NFS poly selection (msieve -np -t <threads>)? one can do this with the new yafu nfs() functionality or by hand, but both ways are not suitable for fully automated aliquot sequence runs with aliqueit.

The factMsieve,pl from http://mersenneforum.org/showthread.php?t=15662 should do what you want. Assuming aliqueit can generate a NAME.n file and call factMsieve.pl then it should do the rest once you've set paths to binaries, number of CPUs, etc.

The (untested) facility to start helpers automatically was designed for aliqueit etc to exploit multiple systems. I've just not had time to set up passwordless ssh (rsh or rexec should work instead of ssh) from the master to a helper and test it.

Chris K

Andi47 · 2011-06-18, 16:57

Quote:

Originally Posted by chris2be8

The factMsieve,pl from http://mersenneforum.org/showthread.php?t=15662 should do what you want. Assuming aliqueit can generate a NAME.n file and call factMsieve.pl then it should do the rest once you've set paths to binaries, number of CPUs, etc.

The (untested) facility to start helpers automatically was designed for aliqueit etc to exploit multiple systems. I've just not had time to set up passwordless ssh (rsh or rexec should work instead of ssh) from the master to a helper and test it.

Chris K

Thanks, I will check it out.

Andi47 · 2011-06-18, 18:31

Quote:

Originally Posted by Andi47

Thanks, I will check it out.

Poly search seemed to run fine (although I did no deep check if it in fact did 6-threaded poly search), but when sieving was about to start, I got this error:

Code:

syntax error at factMsieve.pl line 103, near "my "
Global symbol "$LARGEP" requires explicit package name at factMsieve.pl line 103.
Global symbol "$LARGEP" requires explicit package name at factMsieve.pl line 186.
BEGIN not safe after errors--compilation aborted at factMsieve.pl line 668.

reverting to older but stable version of factmsieve.pl

Edit: I changed my $NICE="nice -n 19 "; to my $NICE=""; hence nice does not work under windows (gives an error "statement is spelled wrong or does not exist"). Could this have caused the problem? (my $NICE= " " did'nt work either.)

chris2be8 · 2011-06-19, 15:02

Strange, the code works for me, but I run Linux. Are you sure you didn't delete the ; on the end of the my $NICE= line? If I deliberately delete it I get similar error messages.

Chris K

chris2be8 · 2011-06-19, 15:13

When using msieve for line sieving under what circumstances does it write the relations in compressed format? I want to run msieve's line sieving to a new file which I can then copy into the $NAME.dat file, but I'm getting compressed relations which can't be copied into the same file as the uncompressed relations from gnfs-lasieve4I*

I managed to get it to work by renaming the output file to end with .gz, gunzip'ing it and copying that into $NAME.dat, but thats not very likely to work on Windows systems which probably won't have gunzip installed.

Chris K

Andi47 · 2011-06-19, 15:28

Quote:

Originally Posted by chris2be8

Are you sure you didn't delete the ; on the end of the my $NICE= line? If I deliberately delete it I get similar error messages.

Chris K

ooops you are right, the ; went indeed missing. I will do another test run in a few days. (I don't want to interrupt my current factorization.)

chris2be8 · 2011-06-21, 16:44

Quote:

Originally Posted by chris2be8

When using msieve for line sieving under what circumstances does it write the relations in compressed format? I want to run msieve's line sieving to a new file which I can then copy into the $NAME.dat file, but I'm getting compressed relations which can't be copied into the same file as the uncompressed relations from gnfs-lasieve4I*

Any comments? Jason? Anyone?
Chris K

jasonp · 2011-06-21, 17:01

Msieve uses an abstraction to represent a source for reading or writing relations. If compiled with NO_ZLIB=1, relation input and output are assumed to be of uncompressed text. Otherwise the same algorithm from gzip is used automatically to read from and write to the file, and thus the whole file has to be compressed (or not). Note that you can concatenate compressed relation files with no further processing, and the result is still a valid compressed file.

(My windows binaries are always compiled with NO_ZLIB=1)

chris2be8 · 2011-06-22, 17:04

Does it always write uncompressed data if the output file already contains some uncompressed data? That's the only difference I can see in my two cases.

Also does msieve object if there is more that 1 "N " line in the input .dat file?

Recompiling with NO_ZLIB=1 won't help, I'm trying to update factMsieve.pl for anyone to use and I can't guarantee everyone will do that (or even know how to).

Chris K

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve 1.53 feedback	xilman	Msieve	149	2018-11-12 06:37
Msieve 1.50 feedback	firejuggler	Msieve	99	2013-02-17 11:53
Msieve 1.43 feedback	Jeff Gilchrist	Msieve	47	2009-11-24 15:53
Msieve 1.42 feedback	Andi47	Msieve	167	2009-10-18 19:37
Msieve 1.41 Feedback	Batalov	Msieve	130	2009-06-09 16:01

2011-06-18, 10:56	#2
jasonp Tribal Bullet Oct 2004 3,541 Posts	For at least the next few months, I'm only going to have the time to do some deep explorations on running stage 2 for very large problems. After that, I've been thinking of an overhaul of the code to use some of the nice stuff in YAFU.

2011-06-19, 15:02	#6
chris2be8 Sep 2009 2·1,039 Posts	Strange, the code works for me, but I run Linux. Are you sure you didn't delete the ; on the end of the my $NICE= line? If I deliberately delete it I get similar error messages. Chris K

2011-06-19, 15:13	#7
chris2be8 Sep 2009 100000011110₂ Posts	When using msieve for line sieving under what circumstances does it write the relations in compressed format? I want to run msieve's line sieving to a new file which I can then copy into the $NAME.dat file, but I'm getting compressed relations which can't be copied into the same file as the uncompressed relations from gnfs-lasieve4I* I managed to get it to work by renaming the output file to end with .gz, gunzip'ing it and copying that into $NAME.dat, but thats not very likely to work on Windows systems which probably won't have gunzip installed. Chris K

2011-06-21, 17:01	#10
jasonp Tribal Bullet Oct 2004 3,541 Posts	Msieve uses an abstraction to represent a source for reading or writing relations. If compiled with NO_ZLIB=1, relation input and output are assumed to be of uncompressed text. Otherwise the same algorithm from gzip is used automatically to read from and write to the file, and thus the whole file has to be compressed (or not). Note that you can concatenate compressed relation files with no further processing, and the result is still a valid compressed file. (My windows binaries are always compiled with NO_ZLIB=1)

2011-06-22, 17:04	#11
chris2be8 Sep 2009 2×1,039 Posts	Does it always write uncompressed data if the output file already contains some uncompressed data? That's the only difference I can see in my two cases. Also does msieve object if there is more that 1 "N " line in the input .dat file? Recompiling with NO_ZLIB=1 won't help, I'm trying to update factMsieve.pl for anyone to use and I can't guarantee everyone will do that (or even know how to). Chris K