mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > NFS@Home

Reply
 
Thread Tools
Old 2020-08-31, 02:39   #1
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41·109 Posts
Default Let's change the NFS@home sievers!

After nearly 10 years in service, the 14e queue has outlived its usefulness. Most of the workunits (not jobs) in 14e are for numbers that would be faster on 15e, but the 15e queue is lengthy and the 14e queue often runs dry, so they're sent to 14e somewhat wastefully.

So, why don't we retire 14e, and change lasieved to the 15e siever? In order to maintain three BOINC projects within NFS@home, I have a couple of ideas:

1. lasieved w/15e jobs should have lim's restricted to maintain a low-memory option for users on older machines. Limits of 134/200M or 134/134M would keep memory use about where it is now (note some 14e jobs use lim's of 268M!) Note this limit would not be hard-coded, so we could let it rise over the years as jobs and memory capacities get bigger.

2.If we change d to 15e, the lasieve-e BOINC project should change to 16e (or, failing that, 15e -J 15). Many of the current "e" jobs would be more efficient on 16e, and the queue is long in part because of these huge jobs. Further, there are quite a few GNFS-190+ or SNFS-275+ jobs of interest among the projects we collectively care about, so this queue would stay well-fed and yet faster than the current arrangement (since each job of that size is faster on 16e than 15e).

For me, the second change is not required by the first- we could have 15e-small and 15e-big for the two queues and still come out more efficient overall.

Note that going to larger sievers also tends to make smaller matrices for each job, or equivalently require somewhat fewer relations to yield a matrix.

Upsides:
1. Lots of small-15e jobs are currently homeless, either waiting months in the 15e queue or wasting resources on 14e. These jobs are very common in our work now, and the jobs continue to grow over time. It's worth it to change now, and will become more valuable in the future.

2. We have the hardware to handle matrices of 50M size, and using 16e would allow us to handle up to GNFS-205 or SNFS-300 (? Not sure, I don't do much SNFS) without troubling Greg on the big queue.

Downsides:
1. Some small jobs, e.g. 14e with 30LP or smaller, will be less efficient on 15e than 14e. This computational waste is much less than the savings available from jobs currently run as 14e/32, e.g. GNFS-175ish. The really small ones (e.g.. GNFS 155 or less) can be done on a home desktop anyway.

2. Memory use might rise on the "e" queue. We should be careful to keep 16e lim's small at the outset to make sure memory use doesn't drastically change if we do this.

3. If "d" queue has a strict lim cap, there will still be a size of job that's homeless, with 16e slow but 15e/134M too restrictive. We can handle this as it happens, likely by sending to whichever queue is shorter.

4. Greg might not have time nor interest in the effort required to make this change happen.

Thoughts?
VBCurtis is offline   Reply With Quote
Old 2020-08-31, 07:46   #2
xilman
Bamboozled!
 
xilman's Avatar
 
"𒉺𒌌𒇷𒆷𒀭"
May 2003
Down not across

1019110 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
...

Thoughts?
I would support this.

Some time ago NFS@home ran a number of 14e SNFS for me when there will still suitable candidates. After they dried up I stopped submitting because the 15e queue was even then over-subscribed.
xilman is offline   Reply With Quote
Old 2020-08-31, 08:14   #3
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

3·37·43 Posts
Default

Seconded.
The only recommendation is to also update the memory requirements on the project, since beginning people complain about this, just take a look at the forum. 14e using more memory than 15e and 16e, or 15e more than 16e.
pinhodecarlos is online now   Reply With Quote
Old 2020-08-31, 12:53   #4
swellman
 
swellman's Avatar
 
Jun 2012

22×3×241 Posts
Default

I’m all for such changes.
swellman is online now   Reply With Quote
Old 2020-08-31, 15:55   #5
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

10001011101012 Posts
Default

Quote:
Originally Posted by pinhodecarlos View Post
Seconded.
The only recommendation is to also update the memory requirements on the project, since beginning people complain about this, just take a look at the forum. 14e using more memory than 15e and 16e, or 15e more than 16e.
Carlos-
As our resident BOINC-people expert, what should be the max memory use for the "d" subproject, and for "e"? I can work out maximum lim's on 15e for "d" from your suggestion.

Or, should we go about it the other way, measuring RAM use for a typical new "d" job and then claim memory needs are a little higher than that?

What do the rest of you think of a cap at GNFS-185 for the new "d"? That can be run with lim's of 134M. What would a corresponding SNFS difficulty be? 275 (assuming small sextic-poly coefficients)?

If frmky doesn't reply here within a week or so, I'll PM him about this idea before I get too carried away with memory measurements/etc.
VBCurtis is offline   Reply With Quote
Old 2020-08-31, 16:24   #6
jyb
 
jyb's Avatar
 
Aug 2005
Seattle, WA

5×11×29 Posts
Default

Yes, I believe the time has come for this change. I support it. Thank you for getting the ball rolling, Mike.
jyb is offline   Reply With Quote
Old 2020-08-31, 16:26   #7
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

3·37·43 Posts
Default

Curtis,

Beyond memory requirements there’s also the credits requirements and badge requirements. Everything is related, in the beginning Greg wanted to credit more for the app using more memory (and higher difficulty jobs) the 16e, but when you guys started to run 32-bit jobs with higher lims on the 14e all started to fall apart, that’s when the complains started.

Badge lower granularity is also a problem, it is impossible for comum mortals to achieve them, only with computer farms.

Regarding memory requirements what I’m seeing is that the number of cores increase is not
followed by more RAM per core. It’s stalled to 2GB/core, so I would say you could at the limit double the memory requirements as they are now stated on the projects details. I would suggest the following but it is better to request Greg to query the database to see what’s the average memory available per core.

-d version to 0.75 GB
-e version to 1.25 GB
-f version to 1.75 GB

Last fiddled with by pinhodecarlos on 2020-08-31 at 16:28
pinhodecarlos is online now   Reply With Quote
Old 2020-08-31, 16:34   #8
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

3·37·43 Posts
Default

Everything needs to be balanced, it is not only about the memory. It’s a trade off between memory usage, wu size, wu processing timings, level of granularity of the available badges.
pinhodecarlos is online now   Reply With Quote
Old 2020-09-01, 07:52   #9
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

22×33×19 Posts
Default

Would all of the needs be met by the creation of lasievee_small (15e smaller numbers) and lasievef_small (16e smaller numbers) queues? On the back end these would be the same as the existing lasievee and lasievef queues, but guidelines for size and memory requirements for submitted jobs could be determined by the community and evolve over time as needed.

These new queues could have their own separate stats and badges, or could be combined with the existing queues' stats (d+e_small and e+f_small for example). Carlos?
frmky is offline   Reply With Quote
Old 2020-09-01, 09:03   #10
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

12A516 Posts
Default

Quote:
Originally Posted by frmky View Post
Would all of the needs be met by the creation of lasievee_small (15e smaller numbers) and lasievef_small (16e smaller numbers) queues? On the back end these would be the same as the existing lasievee and lasievef queues, but guidelines for size and memory requirements for submitted jobs could be determined by the community and evolve over time as needed.

These new queues could have their own separate stats and badges, or could be combined with the existing queues' stats (d+e_small and e+f_small for example). Carlos?
Separate queues, separate badges otherwise clients will not run it for own milestones.
pinhodecarlos is online now   Reply With Quote
Old 2020-09-01, 15:00   #11
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41·109 Posts
Default

Quote:
Originally Posted by pinhodecarlos View Post
Separate queues, separate badges otherwise clients will not run it for own milestones.
Is this true even if the old d and e queues cease to exist?

That is, if the 15e-small replaces d and counts for d badges, they would still stop running it?

Doesn't matter to us queue-feeders what badges are given, I merely wish to make sure we're talking about the same thing.
VBCurtis is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
ECM change Prime95 PrimeNet 28 2020-09-02 08:16
Compiling GNFS sievers on AArch64 platform wombatman Programming 11 2017-03-11 03:12
gnfs asm version sievers illegal instruction EdH Factoring 32 2016-10-12 20:49
Name Change? Fred Lounge 8 2016-01-31 17:42
Calling all 64-bit Linux sievers! frmky NFS@Home 25 2013-10-16 15:58

All times are UTC. The time now is 21:51.

Tue Nov 24 21:51:51 UTC 2020 up 75 days, 19:02, 4 users, load averages: 3.28, 3.13, 3.13

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.