Sieving drive for k=1003-2000 n=500K-1M

[PCZ]

I reckon we should work out a credit system for sieving.
Not because i want the points but because others will do it for the points.

If this project gets more crunchers then some of us might end up 24/7 365 sieving.

Now i dont care for boinc myself but if the sieving was done as a boinc project i wouldn't mind.

As long as they dont do any LLR tests :)

IB aren't you all sieved out yet, cos you soon will be.

[IronBits]

Heck ya! Want my ranges? :p
have yoyo do the wrapper for NPLB Sieving! :D

[mdettweiler]

Originally Posted by PCZ

I reckon we should work out a credit system for sieving.
Not because i want the points but because others will do it for the points.

If this project gets more crunchers then some of us might end up 24/7 365 sieving.

Hmm...interesting idea. Though, this brings to mind something I'd been thinking about for a while: before we even think of doing credits/stats for sieving, we should look into adding all the manual (non-LLRnet) LLR work to the DB. It wouldn't be too hard; especially since the DB automatically screens out duplicates (you'll see where this comes in in a moment). Here's what I was thinking:

-First, we could load in all the manual ranges, by sending AMDave CSV files just like is passed around for LLRnet results automatically behind the scenes. For the username we'd of course put the user's ID, and for time/date we'd put the time/date of the forum post they made reporting the range complete. As for the server/port fields, maybe we could load it in as server MN (for manual--unless somebody can come up with something a bit more imaginitive than that ) port 0.

-Then, we could load all the LLRnet results in through the same method (that is, the LLRnet results that have been processed and sent to Gary as manual results), but this time under user "Unknown". That means that any additional k/n pairs that may have "slipped through the cracks" with the servers would be still accounted for.

After doing this, the DB should contain a full accounting of every result returned by the project. Probably one of the biggest benefits of this (besides granting equal credit for manual results as well as automatic ones) would be that the "Drive Progress" scatter plot would be much more well filled in (this would be most noticeable for the 1st and 3rd drives, much of which were done manually).

I remember that I had originally expressed objections to such a plan a while back due to worries about it overloading the DB. Of course, that was back when we had Adam running the DB on his server that could barely tolerate the load anyway, not to mention that all of the page queries were run live (!) and thus some of the pages took quite a long time to load even with just the LLRNet stuff in there. Of course, by now that is no longer a problem--we now have a highly robust server that doesn't seem to have any limitations in sight on the amount of data we can pump into it.

After we've finally addressed the issue of adding manual LLR work into the stats, *then* we can look into crediting sieve work.

[PCZ]

Good idea yoyo can put a wrapper around almost anything.
Sieving is a very manual process though how do you automate it ?
Guess it's been done before ?

[mdettweiler]

Originally Posted by PCZ

Good idea yoyo can put a wrapper around almost anything.
Sieving is a very manual process though how do you automate it ?
Guess it's been done before ?

It has indeed been done before, both by Riesel Sieve BOINC and by PrimeGrid. Probably it wouldn't be too hard to build off of their work (I doubt PrimeGrid would mind sharing their source code; they've done it before with their LLR wrapper).

Of course, if we ever do any sieving work through any sort of BOINC setup, we'd have to make sure of one thing: that whatever project we do it through isn't going to expect us to keep up a steady supply of work. Most of the time, NPLB does *not* have an active sieving drive; in fact, after this one is done, since we'll be all set up to n=1M on both k=300-1001 and k=1003-2000, we probably won't be doing that much more sieving for quite a while (or if we do, it will be as a somewhat minor side effort that we won't be dedicating much resources to for the time being). So, any BOINC projects that we may work with in the future for sieving will have to be prepared to treat our project's portion of their setup as a "standby mode"--i.e., it will be idle most of the time except for those brief periods when we need sieving.

All this considered, however, it is my opinion that at least at this point, doing BOINC even for sieving would be a lot more hassle and effort than it's truly worth (even with an existing BOINC project running it for us). I do believe that BOINC sieving is something to keep on the "back burner" for the future; however, for the time being it would be much preferable to continue to handle our sieving manually.

Max

[PCZ]

Agreed best avoid boinc.
As to manual ranges, i was also thinking that the scripts that parse the servers files could also parse the manual ranges.

[mdettweiler]

Originally Posted by PCZ

As to manual ranges, i was also thinking that the scripts that parse the servers files could also parse the manual ranges.

Well, the problem with that is the fact that LLRNet outputs a bit more data than manual LLR in its results files. Example:

manual LLR
1695*2^549586-1 is not prime. LLR Res64: CD6714BDC076CFC4 Time : 565.193 sec.

LLRnet (on the server end; the client output is more like manual LLR's)
user=mdettweiler
[2009-04-20 03:17:08]
1695*2^549586-1 is not prime. Res64: CD6714BDC076CFC4 Time : 565.193 sec.

To import the LLRnet data into the DB, we have scripts running on the servers that take the server results, convert them to a CSV format, and then plop them in a directory where the DB's scripts will pick them up and import them next time they run.

In order to import manual LLR results, what we'd need to do is code up a variation on the CSV-conversion script that, instead of reading the user and date/time info from the original results file, reads it from (say) the command line. Everything else is common to both manual and LLRnet results file formats.

Max

[IronBits]

I never have liked BOINC, and always loved the creativity of the programmers in designing a client front end.
Something was lost when everything went over to Boinc.
Sure, it's so easy and lame to run, even a caveman could run it.
It's just bland any more. It's all about getting your stats fix, and not so much about a project any more.

[IronBits]

Max, if you want to import all the manual ranges, then, lets get busy with it.
I think it would be great to fill in the database with all ranges.
Something to be said for full accountability.

[henryzz]

if you want more more help in sieving i would suggest a prpnet-like sieving wrapper

[mdettweiler]

Originally Posted by IronBits

Max, if you want to import all the manual ranges, then, lets get busy with it.
I think it would be great to fill in the database with all ranges.
Something to be said for full accountability.

Okay, cool. As soon as Gary gives the OK we can get started.

Gary, if you're OK with this, could you see about sending me all the results for the 1st, 2nd and 3rd drives (to begin with)? I'm pretty sure I could code up a script somewhat quickly for converting these to the standard stats CSV format.