Manual stats import discussion

[gd_barnes]

Originally Posted by Mini-Geek

Sounds like I missed something, probably related to Gary's two posts that he deleted, but I'm kind of glad I missed it because it seems to have been angry and misinformed.

They weren't angry at all. Misinformed perhaps. I was attempting to recommend a plan of action that turned out to not be needed after the phone conversation.

[Mini-Geek]

To have my status in my individual k's counted, will I need to give my results, or will the n I'm at be sufficient? I'd assume the former but would rather check first. Tell me when you're pulling the stats from the individual k drive manual files so I can give the latest results.

Originally Posted by AMDave

Oh, and btw:
PS - the stats site migration is complete

Cool. I'd suggest adding a prominent link (either in the side bar on the left, or somewhere on the home page) to the "start here" sticky once it's created.

[henryzz]

Originally Posted by Mini-Geek

To have my status in my individual k's counted, will I need to give my results, or will the n I'm at be sufficient? I'd assume the former but would rather check first. Tell me when you're pulling the stats from the individual k drive manual files so I can give the latest results.

it will be results i think as i think they store the residues in the database

[mdettweiler]

Originally Posted by henryzz

it will be results i think as i think they store the residues in the database

Until we can get something more automated set up, yes, we will try to get the results imported into the DB as often as they come in. Guys, this means that at least for now, all manual results should have me CCed in when you email them to Gary so that I can get them imported. If you don't have my email address, PM/email me through the forum and I'll send it to you.

[henryzz]

Originally Posted by mdettweiler

If you don't have my email address, PM/email me through the forum and I'll send it to you.

or look at http://bugmesticky.googlepages.com/contactinfo which has his email on(and still lists his mersenneforum username as anonymous)

[IronBits]

[mdettweiler]

Originally Posted by henryzz

or look at http://bugmesticky.googlepages.com/contactinfo which has his email on(and still lists his mersenneforum username as anonymous)

Whoops--I had forgotten to change that. It's fixed now.

[gd_barnes]

To be kind to Max, I would only suggest that manual results be sent at the completion of n=50K or 100K ranges at the very most (if we're talking the individual-k effort). Best of all for the admins would be to wait until you are done with your manual reservation. But I understand how it goes with wanting to see your stats go up. n=50K or 100K ranges is fine.

Max has a tremendous amount of research work involved. I'm sure the automation of the formatting of the results is not a problem for him. It's going to be researching who did what ranges manually from the various drives. I sent him mostly n=20K results ranges for entire drives. They weren't divided up by who did them or whether they were manual. He has to pick out the ones that were manual and who did them before going through the formatting process.

For now, Max is only working on the 1st thru 3rd drives. Please don't bother him with individual-k or other results right now. Those can be sent to me as usual. We'll announce when we're done with manual files on the 1st 3 drives and what drives we are working on next. After that, we'll likely do the 5th thru 10th drives. Those will be a small effort because there were far fewer manual reservations. THEN we'll likely go to the individual-k effort and then perhaps the double-check and mini-drive efforts. The admins haven't discussed if we want to include personal efforts for k>2000 yet. My thinking is yes because several people have put a lot of CPU hours into them and have reported them as NPLB so it seems only fair. But we'll have to have a discussion about it.

The 4th drive results are in a tremendous mess on my end. Although I know where everything is, it's going to be highly difficult to bring it all together and match up with the original sieve file. I hope to get to that someday but don't count on it soon.

I think I got all the drives and efforts there.


Gary

[Mini-Geek]

Originally Posted by gd_barnes

To be kind to Max, I would only suggest that manual results be sent at the completion of n=50K or 100K ranges at the very most (if we're talking the individual-k effort). Best of all for the admins would be to wait until you are done with your manual reservation. But I understand how it goes with wanting to see your stats go up. n=50K or 100K ranges is fine.

If a lot of the work is finding which ranges were done by who, wouldn't sending him results directly (in whatever size chunks it be, unless extremely small) be a lot easier than you giving him an unsorted 20K chunk, because he knows it's all manual and by the one that just sent it to him?

Originally Posted by gd_barnes

Max has a tremendous amount of research work involved. I'm sure the automation of the formatting of the results is not a problem for him. It's going to be researching who did what ranges manually from the various drives. I sent him mostly n=20K results ranges for entire drives. They weren't divided up by who did them or whether they were manual. He has to pick out the ones that were manual and who did them before going through the formatting process.

I could probably help Max out with some of the manual results processing if the process and scripts aren't too complicated and there'd be no problems with a non-admin doing something like that. Just send me a 20K chunk along with any applicable scripts and/or instructions and I'll take a shot at it.

[gd_barnes]

Originally Posted by Mini-Geek

If a lot of the work is finding which ranges were done by who, wouldn't sending him results directly (in whatever size chunks it be, unless extremely small) be a lot easier than you giving him an unsorted 20K chunk, because he knows it's all manual and by the one that just sent it to him?

I could probably help Max out with some of the manual results processing if the process and scripts aren't too complicated and there'd be no problems with a non-admin doing something like that. Just send me a 20K chunk along with any applicable scripts and/or instructions and I'll take a shot at it.

The 20K chunks are sorted by n-value and separated by drive. That's how I store them on my machines. You can imagine how much data I have already. Keeping it in separate n=1K or 2K chunks for manually processed ranges was just too cumbersome to try to match everything up with the original sieve files and verify the primes at various places.

It's a matter of who does what. He's offered to pull out the manual ranges so I just put the big files on my machine for him to copy off and hack away at. I believe he has automated scripts that will take specific n-ranges out of the middle of a larger n-value sorted file. It's likely much faster for him to do it then for me to manually pull out the n-ranges for the manually processed results.

Check with Max and see if he would like some help. If he has the automated "pulling out of manual ranges" process set up already, I suspect that it will be easier for one person to do it then to coordinate and train another person on helping out but I couldn't say for sure.


Gary

[MyDogBuster]

Don't know where this belongs, but I do know that the DB guys read this thread.

The 6k mini drive being run as part of the individual k=300-400 drive is an on-line LLRNET process (GB8000). I don't see the little dots for this on the drive progress chart. It's only 9 primes (so far), but that little corner of the chart is so barren. I think a nice orange will look nice.