mersenneforum.org  

Go Back   mersenneforum.org > Other Stuff > Archived Projects > NFSNET Discussion

 
 
Thread Tools
Old 2006-02-28, 03:38   #1
NeoGen
 
Dec 2005

2·33 Posts
Default 2^1466L done

Another one has reached 100%. This one was much faster than the last one.

What's next on the menu?
NeoGen is offline  
Old 2006-02-28, 15:06   #2
Wacky
 
Wacky's Avatar
 
Jun 2003
The Texas Hill Country

21018 Posts
Default

Really finishing 2,1146L ...
Somebody entered incorrect data for the sieving effort parameters. We are really only about half done.

We are considering doing 2,797+.c150 which is what is left thanks to Bruce finding a 55 digit factor by ECM. This is now in the range where GNFS is more efficient that using the original SNFS polynomials.

However, there is one potential problem for our participants.
The typical SNFS project has something like 40M "lines" where each line is perhaps 90M long. This allows us to make assignments where the atomic assignment takes only a fraction of a minute and each client processes hundreds of them in a single task.

But with the GNFS polynomials, the sieving area is not so square.
Here we need perhaps only 250K lines, but each line is 18.9G long.
The difficulty is that the atomic assignment will now require perhaps 10 minutes, and we have no reasonable way to provide a finer granularity.

For many of the sievers, this is no problem. They sit there and sieve 24/7 at the lowest priority. So, they don't even notice that the OS may have suspended them while something of higher priority was run.

However, we also have another class of participants who frequently start and stop the siever in order to reboot or otherwise use the machine resources. Those users are often willing to wait a few seconds for the siever to stop gracefully. But asking them to wait a number of minutes would not be appropriate.

The best suggestion that I have at the moment is to sieve two projects simultaneously. Those who can process the longer assignments could work on 2,797+ while the others begin another SNFS project.

Comments?

Richard

Last fiddled with by Wacky on 2006-02-28 at 15:07
Wacky is offline  
Old 2006-02-28, 15:17   #3
akruppa
 
akruppa's Avatar
 
"Nancy"
Aug 2002
Alexandria

2,467 Posts
Default

It should be possible to use reciprocal polynomials of opposite skewness, so a line would be only 250k long. This may hurt performance due to very frequent line changes, though.

Alex
akruppa is offline  
Old 2006-02-28, 16:34   #4
NeoGen
 
Dec 2005

2×33 Posts
Default

Quote:
Originally Posted by Wacky
Really finishing 2,1146L ...
Somebody entered incorrect data for the sieving effort parameters. We are really only about half done.
Can it be corrected? If not, it's no problem, but it's the first time I'll see it going to 200%.

Last fiddled with by NeoGen on 2006-02-28 at 16:34
NeoGen is offline  
Old 2006-02-28, 23:17   #5
Jushi
 
Jushi's Avatar
 
Sep 2005
UGent

22·3·5 Posts
Default

Quote:
Originally Posted by Wacky
However, we also have another class of participants who frequently start and stop the siever in order to reboot or otherwise use the machine resources. Those users are often willing to wait a few seconds for the siever to stop gracefully. But asking them to wait a number of minutes would not be appropriate.
How about programming the nfsnetclient to immediatly stop sieving when the user aks to, losing the current line? This way sieving one line could take several minutes, but the program will still stop immediately.

I have to admit I don't know the internals of the code, but I'm sure this is not hard to implement.
Jushi is offline  
Old 2006-03-01, 00:49   #6
Wacky
 
Wacky's Avatar
 
Jun 2003
The Texas Hill Country

32×112 Posts
Default

Quote:
Originally Posted by Jushi
How about programming the nfsnetclient to immediatly stop sieving when the user asks to, losing the current line? This way sieving one line could take several minutes, but the program will still stop immediately.

I have to admit I don't know the internals of the code, but I'm sure this is not hard to implement.
It is more difficult than you think. Because of I/O buffering and "GUI" event loops, you cannot simply "abort" a running thread. In most (OS) cases, you end up having to "poll" to determine if the program should stop. The cost of polling is not insignificant. Therefore, you do not want do do it too often. Finding an appropriate location within the nested loops my not be easy to do.
If it is too low, not only do you waste the time polling, but you may also invalidate the caches, adding even more overhead. On the other hand, if the polling is done too far out, it may not be frequent enough to give satisfactory response. Considering that the length of a "line" may vary by a couple orders of magnitude, finding the right balance is not trivial.
Wacky is offline  
Old 2006-03-01, 09:59   #7
Jushi
 
Jushi's Avatar
 
Sep 2005
UGent

748 Posts
Default

Quote:
Originally Posted by Wacky
It is more difficult than you think. Because of I/O buffering and "GUI" event loops, you cannot simply "abort" a running thread. In most (OS) cases, you end up having to "poll" to determine if the program should stop. The cost of polling is not insignificant. Therefore, you do not want do do it too often. Finding an appropriate location within the nested loops my not be easy to do.
If it is too low, not only do you waste the time polling, but you may also invalidate the caches, adding even more overhead. On the other hand, if the polling is done too far out, it may not be frequent enough to give satisfactory response. Considering that the length of a "line" may vary by a couple orders of magnitude, finding the right balance is not trivial.
I don't know about Windows, but in Unices all this can be done quite easily using signals. You could have one process (nfsnetclient) doing the polling for stop.txt and sending a signal to the linesiever to stop it. The linesiever can then catch this signal to gracefully quit, but maybe this isn't even necessary. Maybe you can simply "abort" a running linesiever.

I know that Linux even has a mechanism using signals to notify when a new file is created in a directory, so then you don't even need to poll for stop.txt (this is probably Linux-specific).
Jushi is offline  
Old 2006-03-01, 13:37   #8
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

1101110101112 Posts
Default

Quote:
Originally Posted by Wacky
The best suggestion that I have at the moment is to sieve two projects simultaneously. Those who can process the longer assignments could work on 2,797+ while the others begin another SNFS project.
What about assigning 'panels', where each client gets maybe 100 lines but only in a specific range?

jasonp
jasonp is offline  
Old 2006-03-01, 15:41   #9
R.D. Silverman
 
R.D. Silverman's Avatar
 
Nov 2003

22×5×373 Posts
Default

Quote:
Originally Posted by jasonp
What about assigning 'panels', where each client gets maybe 100 lines but only in a specific range?

jasonp
This number is small enough so that NFSNET should not be too concerned
about an optimal (or even near-optimal) skewness. Just reduce the line
length and accept some loss in efficiency.
R.D. Silverman is offline  
Old 2006-03-01, 22:52   #10
Wacky
 
Wacky's Avatar
 
Jun 2003
The Texas Hill Country

32·112 Posts
Default

Quote:
Originally Posted by jasonp
What about assigning 'panels', where each client gets maybe 100 lines but only in a specific range?
I have already planned to do that. However, it is logistically difficult to gain even a full order-of-magnitude "improvement" by that technique.

I'll set up a separate server pool and invite those who do not feel that they would be adversely affected by the longer line calculation times to switch. Meanwhile, the rest of the participants can continue sieving as usual.
Wacky is offline  
 

Thread Tools


All times are UTC. The time now is 00:33.


Mon Sep 27 00:33:38 UTC 2021 up 65 days, 19:02, 0 users, load averages: 2.46, 2.69, 2.71

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.