mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > News

Reply
 
Thread Tools
Old 2020-06-20, 03:24   #45
ewmayer
2ω=0
 
ewmayer's Avatar
 
Sep 2002
República de California

23×1,427 Posts
Default

Quote:
Originally Posted by axn View Post
In which case, you'd need to save the current iteration, the last verified GEC check (for rolling back), and the current GEC cumulative product. Still 3 needed.

I'm wondering now how you're managing with just two?
Each redundant p[exponent] and q[exponent] savefile saves current PRP residue and GEC cumulative product. Every 1M iters we do an actual GEC check; if it passes we write both the usual savefiles and a last-good-GEC copy named p[exponent].G. If the next GEC fails, we roll back to the .G file.

I've not yet had that procedure fail, even after a dozen runs on my ultra-flaky (and hence great for testing this stuff) Haswell system, but in the as-yet-unencountered case that the run data and the .G file get corrupted, the user can manually copy the last p[exponent].[k*10]M persistent savefile into both p[exponent] and p[exponent].G and restart from that, maximal loss 10Miters.
ewmayer is offline   Reply With Quote
Old 2020-06-20, 03:27   #46
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

23×3×293 Posts
Default

Latest improvements/updates from the technical discussions.

For an exponent around 100,000,000. We can default to power==7, requiring 1.5GB of disk space during the test, and a 100MB proof file to upload. The server can relatively quickly convert this to a 12.5MB file to store on the server until verification is complete. This 12.5MB file can be sent to a verifier that runs 0.8% of a full PRP test, returning a 256-bit code to the server. Assuming a match, we're all done!

The math looks good that results cannot be faked. The original PRP tester can even be sent the verification work and cannot fake the verification!

The PRP tester could go one power higher which doubles required temp disk space and proof size increases by 12.5MB, verification time is cut in half. Or conversely PRP tester could go one power lower, halving required temp disk space, 12.5MB smaller proof file, and doubling verification time.
Prime95 is online now   Reply With Quote
Old 2020-06-20, 04:03   #47
axn
 
axn's Avatar
 
Jun 2003

2·5·467 Posts
Default

Quote:
Originally Posted by Prime95 View Post
The PRP tester could go one power higher which doubles required temp disk space and proof size increases by 12.5MB, verification time is cut in half. Or conversely PRP tester could go one power lower, halving required temp disk space, 12.5MB smaller proof file, and doubling verification time.
If this is going to be a dedicated worktype outsourced to users, you might as well go one (or even two!!) step lower to reduce the logistics aspect of it.
axn is offline   Reply With Quote
Old 2020-06-20, 05:53   #48
R. Gerbicz
 
R. Gerbicz's Avatar
 
"Robert Gerbicz"
Oct 2005
Hungary

2×13×53 Posts
Default

Quote:
Originally Posted by axn View Post
If this is going to be a dedicated worktype outsourced to users, you might as well go one (or even two!!) step lower to reduce the logistics aspect of it.
With power=5 would you need more than 3% verification time of the original prp test.
R. Gerbicz is offline   Reply With Quote
Old 2020-06-20, 07:55   #49
axn
 
axn's Avatar
 
Jun 2003

123E16 Posts
Default

Quote:
Originally Posted by R. Gerbicz View Post
With power=5 would you need more than 3% verification time of the original prp test.
Which is still 97% better than the 100% verification time today (aka double check). We should optimize for the amount of bandwidth which the authors (i.e. first time testers), verifiers, and the server consume.

Again, someone like Curtis Cooper will feel the pinch when a whole batch of clients suddenly decide to upload gigabytes of data in the middle of the day clogging the university network.

EDIT:- If this ever goes live, we might have to ask Curtis C to change over a significant fraction of their resource to verification work.

Last fiddled with by axn on 2020-06-20 at 07:57
axn is offline   Reply With Quote
Old 2020-06-20, 08:13   #50
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

23·3·293 Posts
Default

Quote:
Originally Posted by axn View Post
Which is still 97% better than the 100% verification time today (aka double check). We should optimize for the amount of bandwidth which the authors (i.e. first time testers), verifiers, and the server consume.

Again, someone like Curtis Cooper will feel the pinch when a whole batch of clients suddenly decide to upload gigabytes of data in the middle of the day clogging the university network.
I suspect a University is least likely to be affected by bandwidth. Temp disk space could be another matter. Random secretary/student/professor will not be happy if prime95 eats up the last few GB of disk space.

I foresee options to tell prime95 when to do uploads, how much disk space each worker can use, and the ability to choose the proof power to some degree. Prime95 default setup should be more multithreaded. 4 workers chewing up 2GB disk each is inferior to one quad-threaded worker using only 2GB total.
Prime95 is online now   Reply With Quote
Old 2020-06-20, 08:37   #51
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

32·5·103 Posts
Default

It’s a good idea setting up Prime95 when to upload since more people will start working from home due to the current circumstances and i already noticed that conference calls by teams and work VPN’s mess up the internet connection even on fibre broadband.
pinhodecarlos is offline   Reply With Quote
Old 2020-06-20, 12:04   #52
ATH
Einyen
 
ATH's Avatar
 
Dec 2003
Denmark

72×59 Posts
Default

Primenet should still store the normal 64 bit residue just in case there is a problem, and someone wants to run a real double check, or years later if someone wants to double check some results when the calculation is a lot faster.
ATH is offline   Reply With Quote
Old 2020-06-20, 12:32   #53
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

23×232 Posts
Default

Quote:
Originally Posted by Prime95 View Post
I suspect a University is least likely to be affected by bandwidth. Temp disk space could be another matter. Random secretary/student/professor will not be happy if prime95 eats up the last few GB of disk space.

I foresee options to tell prime95 when to do uploads, how much disk space each worker can use, and the ability to choose the proof power to some degree. Prime95 default setup should be more multithreaded. 4 workers chewing up 2GB disk each is inferior to one quad-threaded worker using only 2GB total.
Please consider implementing settable download and upload rate limits also. Many universities have fat pipes. And some parts of a university may be modest sized remote sites with thin pipes. I worked at one such for decades. As far as I know, it has never had access faster than 10Mbps. There was a several year stretch of T1 (1.544Mbps). 2GB at T1 is 2.9 hours. At 10Mbs line rate it is 0.44 hours each. Saturating a link can lead to issues for other things, like page loads from a web server. We experienced multiple cases over the years where the incoming or outgoing got saturated and made everyone's internet use difficult or fruitless for the duration. (Third party IP spoofed packet to cause a Swiss HP printer to produce a flood to us, or someone opening a malicious email attachment that launched a spam server, that sort of thing.) Periodically creating sustained link saturation may get the activity banned from a network.

User settable:
  • schedule window for downloads of verification work
  • schedule window for uploads of proof files
  • transmission rate limit(s); separate up and down for asymmetric connections?
  • disk quota
  • participation in verification work
  • proof power preference

Last fiddled with by kriesel on 2020-06-20 at 12:44
kriesel is offline   Reply With Quote
Old 2020-06-20, 12:52   #54
preda
 
preda's Avatar
 
"Mihai Preda"
Apr 2015

1,153 Posts
Default

Quote:
Originally Posted by ewmayer View Post
Here gpuowl - this one is also for a PRP tests but is clearly just of single-residue size, so gpuowl must stash the GEC residue somewhere else ... but the 104960929-old.owl file is the only other one of that size I see. Ah, I get it - gpuowl is storing last-GEC-good residue in the 'old' file, and latest PRP-test residue in the non-old file, and using an on-the-fly method to generate a GEC residue at each restart, rather than a whole-run-cumulative one as I use:
I have to correct that interpretation of gpuowl's checkpoint size: gpuowl does store a single residue, and that is just the GEC (the "from the start" one). The actual residue can be computed from the GEC residue (but not the other way around), and this is done on load.

Last fiddled with by preda on 2020-06-20 at 12:54
preda is online now   Reply With Quote
Old 2020-06-20, 13:05   #55
preda
 
preda's Avatar
 
"Mihai Preda"
Apr 2015

1,153 Posts
Default

Quote:
Originally Posted by axn View Post
In which case, you'd need to save the current iteration, the last verified GEC check (for rolling back), and the current GEC cumulative product. Still 3 needed.

I'm wondering now how you're managing with just two?
Just one! gpuowl only ever saves verified residues, and only writes the base-3 GEC residue, and the res64 of the normal residue. On load the residue is recomputed from the GEC and res64 compared as a check.
preda is online now   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Your help wanted - Let's buy GIMPS a KNL development system! airsquirrels Hardware 313 2019-10-29 22:51
Is GMP-ECM still under active development? mathwiz GMP-ECM 0 2019-05-15 01:06
LLR 3.8.6 Development version Jean Penné Software 0 2011-06-16 20:05
LLR 3.8.5 Development version Jean Penné Software 6 2011-04-28 06:21
LLR 3.8.4 development version is available! Jean Penné Software 4 2010-11-14 17:32

All times are UTC. The time now is 01:28.

Wed Aug 12 01:28:43 UTC 2020 up 25 days, 21:15, 1 user, load averages: 2.50, 2.07, 1.90

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.