BOINC NFS sieving - NFS@Home

[debrouxl]

Just to clear a possible misunderstanding (not necessarily for you): note that I suggested (or at least meant to suggest) adding you to the list of people with access to NFS@Home's work feeding infrastructure. I didn't mean "boo hoo, I'm going away if I don't have my way, and leave you to pick the pieces" - which would be very childish a behaviour from me (and besides, it would be incoherent with the beginning of the paragraph)


I hear your concern, and I hope you've heard my arguments.
What do others think about changing the amount of sieving (or not doing so) ? In fact, it probably boils down to "do we want to sieve more numbers, or make post-processers happier" ?

[fivemack]

Tue May  7 06:56:15 2013  RelProcTime: 19243
Tue May  7 06:56:22 2013  initialized process (0,0) of 3 x 8 grid
Tue May  7 08:07:04 2013  matrix is 15046947 x 15047129 (4521.9 MB) with weight 1325298044 (88.08/col)
Tue May  7 08:21:37 2013  commencing Lanczos iteration
Sun May 12 08:56:47 2013  lanczos halted after 237860 iterations (dim = 15041465)
Mon May 13 20:18:47 2013  sqrtTime: 8668
Mon May 13 20:18:47 2013  prp111 factor: 241215750038856263159716121686098664477752682525771365953523520412080494368158775294634138935517800635218113117
Mon May 13 20:18:47 2013  prp115 factor: 1148844274603906293890529267369286595873870345990523419368092555061370174996362324477929708660691592098869471164329

[R.D. Silverman]

Originally Posted by wblipp

Are you claiming that oversieving to get smaller, easier solved matrices is pushing the state of the art? If not, then you are completely irrelevant to the discussion of how to best tap the BOINC power. I think the issues are social, not mathematical.

You need to learn how to read. The OP explicitly stated
that the work was advancing mathematical knowledge.

[R.D. Silverman]

Originally Posted by debrouxl

Just to clear a possible misunderstanding (not necessarily for you): note that I suggested (or at least meant to suggest) adding you to the list of people with access to NFS@Home's work feeding infrastructure. I didn't mean "boo hoo, I'm going away if I don't have my way, and leave you to pick the pieces" - which would be very childish a behaviour from me (and besides, it would be incoherent with the beginning of the paragraph)


I hear your concern, and I hope you've heard my arguments.
What do others think about changing the amount of sieving (or not doing so) ? In fact, it probably boils down to "do we want to sieve more numbers, or make post-processers happier" ?

Neither one contributes (as you claimed) towards
" advancing mathematical knowledge".

If you believe otherwise, then please show us the advances in mathematics
that have been made.

[fivemack]

Yes, in some sense the OP was wrong. We know this. Repeatedly pointing it out doesn't help.

We're not doing cutting-edge mathematical research here, and we know it; this particular problem is much closer to operations research, how best to parcel out cheap and expensive resources to get as many cubic fathoms of ore as possible out of the mine.

It's not very interesting ore, it won't make Math: Comp: let alone a heavier-duty journal. We know that, too. But it's still a worthwhile problem for us to attack, even if the answer is that it depends entirely on hard-to-measure parameters of how much work is perceived to cost.

[frmky]

The 14e and 15e sieves are simply hobbies at this point. The larger 16e work I feel remains quite useful. It has directly led to the development and extensive testing of better parallel LA tools using MPI, and we are starting to explore using modern GPUs for LA. I agree none of this is a fundamental theoretical breakthrough, but I believe it is an important advance.

Back to Carlos' original question, his data actually tell me that the 15e numbers may be too oversieved. As long a LA resources are plentiful, adding 50% more time to a single run isn't significant. That time could be spent sieving additional numbers.

[wblipp]

Originally Posted by R.D. Silverman

You need to learn how to read. The OP explicitly stated that the work was advancing mathematical knowledge.

I can read just fine. You need to learn how to read context. Sure it was a mis-step to argue that the work was advancing mathematics. But the debate was about whether to oversieve with BOINC resources. If you aren't pointing out the mis-step in order to argue the counter point, what exactly IS your point in the context of this debate?

Perhaps more interesting, what is you position on the question? You've done some interesting work about optimal resource allocation in the simpler environment of homogenous rosources and only ECM - do you have thoughts about how to frame this question in this environment - perhaps even thoughts about how to quantify an objective function? In the statement you objected to, debrouxl was arguing that the objective function should be maximizing the factoring rate. pinhodecarlos is arguing the objective function should be minimizing post processing time because that serves as a proxy for an objective function of maximizing happiness of all participants. what say you?

[fivemack]

May I take L1809?

[frmky]

Originally Posted by fivemack

May I take L1809?

It will be a week or two before it's ready, but sure, it's yours.

[pinhodecarlos]

Originally Posted by wblipp

In the statement you objected to, debrouxl was arguing that the objective function should be maximizing the factoring rate. pinhodecarlos is arguing the objective function should be minimizing post processing time because that serves as a proxy for an objective function of maximizing happiness of all participants. what say you?

Let's all look again to these figures, LA ETA estimates on a Ivy Bridge laptop processor running 4 threads:

L1201
218-digit input - lasievee
SNFS 251.0
setting target matrix density to 110.0
found 74947407 hash collisions in 314763320 relations
found 80865618 duplicates and 235116244 unique relations
matrix is 10065336 x 10065561 (4163.7 MB) with weight 1053317911 (104.65/col)
sparse part has weight 990830628 (98.44/col)
linear algebra at 0.0%, ETA 161h38m

L1803
216-digit input - lasievee
SNFS 251.2
setting target matrix density to 110.0
found 74270715 hash collisions in 329955633 relations
found 76882467 duplicates and 253885458 unique relations
matrix is 10657840 x 10658088 (2884.9 MB) with weight 695106766 (65.22/col)
sparse part has weight 649681132 (60.96/col)
linear algebra at 0.0%, ETA 128h26m

11_10_233m
183-digit input - lasieved
SNFS 244
setting target matrix density to 110.0
found 39972611 hash collisions in 217657588 relations
found 39223437 duplicates and 179652801 unique relations
matrix is 11770611 x 11770836 (4927.7 MB) with weight 1263586357 (107.35/col)
sparse part has weight 1174068997 (99.74/col)
linear algebra at 0.0%, ETA 229h50m

11_10_233m was clearly undersieved. I am in favor of having at least 200-210 u.r for jobs below SNFS difficulty of 250.
Post-processing jobs of 14e and 15e sieves are one man job while 16e sieves are cluster jobs so I think we need to reach a optimal agreement on how much more or less sieve is needed.
Sieving more don't hurt the sievers due to the fact that they don't care about wasting more energy on this task, they just care about the points. If they don't care about it I do care about minimizing the post-processing time, therefore wasting less energy. Do you guys see the sievers on NFS@Home forum complaining that a task is oversieved? They don't understand the math behind NFS@Home....duh! Let's use the sieve power available to make more tests on how much sieve is needed for a certain type of integer and SNFS/GNFS difficulty or lets push the 16e siever.

Ok, probably 15e sieves are a little oversieved but I enjoy doing two post-processing jobs in less than a week on my laptop.

Carlos

[pinhodecarlos]

Originally Posted by frmky

...and we are starting to explore using modern GPUs for LA.

Can you elaborate more? NVIDIA or AMD cards?

Carlos