10^263-1 sieving - Page 5

J.F. · 2009-01-20, 21:35

Yep, a couple of shell scripts around qsub makes life pretty easy.

To illustrate: I work on a 200 node cluster, each a dualcore 2Ghz Opteron. I chose to take jobs to be 1e5 large, which are automatically split over both cores (this was actually a bit tricky to get to work).

To fire 100 jobs in the queue I just type:
~/factoring/runduals.sh poly.txt -a 900 999
This script does something like

Code:

for block in $blockindices; do 
   ~/factoring/submit_dual_blocks.sh $polyfile $ratalg $block $projectname
done

The real work is done in submit_dual_blocks.sh:
* generate poly file from template
* generate job file, tailored to dualcores
* submit

At any moment a postprocessing script can be executed to bake the pancakes which have all ingredients in place.
Uploading (with lftp) is still a manual action, don't really need to automate that.

If anyone is interested, I could place the files online somewhere.

fivemack · 2009-01-23, 00:18

OK, we've now got 100MQ collected and the shape of the relations-against-Q curve is pretty clear.

Would anyone be able to run 15-25 both sides? My 16 cores are quite solidly committed, but it looks as if I misplaced the point of diminishing sieving returns and it's worth doing these small Q.

Xyzzy · 2009-01-23, 00:33

This isn't what you are asking for, but we will have 25-30M done in a few days.

Xyzzy · 2009-01-26, 05:05

Quote:

Would anyone be able to run 15-25 both sides?

We can run the "A" side for 15-20, but it will take about 10 days. Is that acceptable?

Did the new upload for a25-30.bz2 work? We renamed the corrupted file with a "-" at the end and also included a MD5 checksum for the new file.

Xyzzy · 2009-01-26, 19:37

We started the "A" side for 15-20 last night. If this is not cool just let us know.

fivemack · 2009-01-26, 20:35

A 15-20 sounds excellent; I'll take both sides 20-25 and leave R15-20 for some enterprising individual.

J.F. · 2009-01-28, 18:19

Quote:

Originally Posted by fivemack

... and leave R15-20 for some enterprising individual.

That would be me, I guess.

fivemack · 2009-01-30, 18:30

I forgot that the partition for collecting relations was of finite size and that I had left the complete sets of relations for two large SNFS projects on it, so the disc filled up. There is now 17G of free space, which ought to suffice.

Andi47 · 2009-02-04, 09:21

What is the current status - how many relations do we have collected so far? Are more ranges needed to be sieved?

BTW: my range is 99.85% done and should finish today - I can upload it tonight or tomorrow in the evening.

fivemack · 2009-02-04, 11:21

I think most users would be best to go onto the polynomial search for 109!+1 now; we've got 150MQ searched, I'll build a matrix once I've finished 20-25 and xyzzy has finished 15-20, and if the matrix is really ugly I'll politely beseech J.F. to do another 10MQ.

I've ordered a new PSU for the matrix-solving box that broke, but there may be a further delay if the PSU's death killed the motherboard. I am unsure as to the net economic disadvantage of cheap but unreliable PSUs; losing the computer for two days every six months is only a 1% slowdown, and I wouldn't have paid the £35 difference between a £15 and a £50 PSU in order to make the computer 1% faster.

Andi47 · 2009-02-04, 19:59

my pancakes should have reached Oslo.

2009-01-20, 21:35	#45
J.F. Jun 2008 2³×3² Posts	Yep, a couple of shell scripts around qsub makes life pretty easy. To illustrate: I work on a 200 node cluster, each a dualcore 2Ghz Opteron. I chose to take jobs to be 1e5 large, which are automatically split over both cores (this was actually a bit tricky to get to work). To fire 100 jobs in the queue I just type: ~/factoring/runduals.sh poly.txt -a 900 999 This script does something like Code: for block in $blockindices; do ~/factoring/submit_dual_blocks.sh $polyfile $ratalg $block $projectname done The real work is done in submit_dual_blocks.sh: * generate poly file from template * generate job file, tailored to dualcores * submit At any moment a postprocessing script can be executed to bake the pancakes which have all ingredients in place. Uploading (with lftp) is still a manual action, don't really need to automate that. If anyone is interested, I could place the files online somewhere. Last fiddled with by J.F. on 2009-01-20 at 21:48 Reason: added details

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
NFS sieving?	Dubslow	Factoring	8	2012-09-28 06:47
Line sieving vs. lattice sieving	JHansen	NFSNET Discussion	9	2010-06-09 19:25
10^420 + 1 sieving	juno1369	Factoring	20	2010-04-28 01:11
Sieving	OmbooHankvald	Prime Sierpinski Project	4	2005-06-30 07:51
Sieving	robert44444uk	Sierpinski/Riesel Base 5	8	2005-04-02 22:30

2009-01-23, 00:18	#46
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 6424₁₀ Posts	OK, we've now got 100MQ collected and the shape of the relations-against-Q curve is pretty clear. Would anyone be able to run 15-25 both sides? My 16 cores are quite solidly committed, but it looks as if I misplaced the point of diminishing sieving returns and it's worth doing these small Q.

2009-01-23, 00:33	#47
Xyzzy "Mike" Aug 2002 5·17·97 Posts	This isn't what you are asking for, but we will have 25-30M done in a few days.

2009-01-26, 19:37	#49
Xyzzy "Mike" Aug 2002 5·17·97 Posts	We started the "A" side for 15-20 last night. If this is not cool just let us know.

2009-01-26, 20:35	#50
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 2³·11·73 Posts	A 15-20 sounds excellent; I'll take both sides 20-25 and leave R15-20 for some enterprising individual.

2009-01-30, 18:30	#52
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 1918₁₆ Posts	I forgot that the partition for collecting relations was of finite size and that I had left the complete sets of relations for two large SNFS projects on it, so the disc filled up. There is now 17G of free space, which ought to suffice.

2009-02-04, 09:21	#53
Andi47 Oct 2004 Austria 4662₈ Posts	What is the current status - how many relations do we have collected so far? Are more ranges needed to be sieved? BTW: my range is 99.85% done and should finish today - I can upload it tonight or tomorrow in the evening.

2009-02-04, 11:21	#54
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 2³·11·73 Posts	I think most users would be best to go onto the polynomial search for 109!+1 now; we've got 150MQ searched, I'll build a matrix once I've finished 20-25 and xyzzy has finished 15-20, and if the matrix is really ugly I'll politely beseech J.F. to do another 10MQ. I've ordered a new PSU for the matrix-solving box that broke, but there may be a further delay if the PSU's death killed the motherboard. I am unsure as to the net economic disadvantage of cheap but unreliable PSUs; losing the computer for two days every six months is only a 1% slowdown, and I wouldn't have paid the £35 difference between a £15 and a £50 PSU in order to make the computer 1% faster.

2009-02-04, 19:59	#55
Andi47 Oct 2004 Austria 2×17×73 Posts	my pancakes should have reached Oslo.