-   No Prime Left Behind (
-   -   Manual stats import discussion (

gd_barnes 2009-05-03 01:54

Manual stats import discussion
I have some exciting news:

We have now begun the long task of converting and loading manually processed results into our stats database.

See details in [URL=""]this post[/URL].


MyDogBuster 2009-05-03 02:04

[QUOTE]We have now begun the long task of converting and loading manually processed results into our stats database.

Yeah. This will give Free-DC the CORRECT numbers to shoot for in overtaking ROLP. Now the numbers will mean something besides just the LLRNET totals.

IronBits 2009-05-03 03:26

Let me get this straight...
Basically, you guys got 1+ years head start before I even heard of NPLB.
I took over the whole project, server wise, when something happened and the project was in jeapordy.
I managed to get Bok and AMDave interested in assisting me with the database, while I cranked out code under Windows to make it all work.
Then AMDave started busting his tail to get everything migrated over to Linux and made all kinds of enhancements, and Bok added NPLB stats to his stats system.
With the new stats system being developed and working rather well...
You started a Team called ROLP, some of you banned together...
Then ROLP got stomped by Free-DC in a short rally, then all of a sudden, the stats system isn't good enough because it was only reflecting how many knpairs were being returned and some credit system had to be developed.
All good ideas and everything is now moving along nicely.
Another rally was called for and Free-DC declared another BBQ run, and whooped ROLP again.

Free-DC is now catching you so quickly in the daily Server stats, even with the new stats system, and I noticed more of the old timers join ranks with ROLP (nothing wrong with that, gives Free-DC more to shoot for)

Now you are adding in all the manual work and information into the database, which is a great idea!
Keeping track of who did what work is very important and a fantastic idea!

Somehow, the mixing of the manual results scores into the active server results scores, especially considering the lead time you guys had, is not such a good idea.
There is no doubt in my mind what-so-ever that you 'old timers' have put a lot of time and effort into the project, even before I heard about the project, and continued to do so even today with manual reservations.

Everyone watches the daily active stats, and when the last few persons joined ROLP, the stats took a big jump for those days, no biggy...

But, to add in all the manual reservations on top of that, and then tell everyone that is the true scores, even though they are very accurate,
is not going to go over very well with the majority of folks that have been watching the stats.
Especially Free-DC.
The target was set on May 1st.
By May 31st, we should catch ROLP, or be very very close, but now the whole Team will be demoralized as you keep adding in more scores for work you have done in the past that they never knew about...
I can't think any more and I'm sure you get my point.
I totally understand what's going on, agree with what you need to do to get it all into the database.
Problem is what to do about offline crunching versus online crunching.
There is an unfair (if you can call it that) advantage to the old timers in that they know how to do go about fuzzing about with manual reservations, getting more done with optimized clients that work faster than the clients used to work online crunching, which is what the majority of new users are going to be...

Simple version.
Started teams and started rallies to get the project moving
Challenges were made by ROLP.
Free-DC easily answers the call, beats ROLP.
Stats scoring system developed, ROLP takes a huge jump in the lead (no problem)
Another Rally is declared
Free-DC answers the call again and beats ROLP.
New challenge issued
Free-DC declares that the Project of the Month is going to be NPLB
Gap starts closing rapidly, old timers on the side lines join ranks with ROLP, one even jumped team affiliation (fine, no problem)
Now offline crunching will be added to the database (understandably so)

I feel like a carrot is being dangled just beyond my reach...


MyDogBuster 2009-05-03 03:50

[QUOTE]Somehow, the mixing of the manual results scores into the active server results scores, especially considering the lead time you guys had, is not such a good idea.[/QUOTE]

There are a lot of manual files done by Free-DC folks too. Carlos has a ton of them. I just think it's a truer representation of what we have done as a project.

I would have no problem showing the manual results seperately with maybe a summary showing everything combined. My only concern is to get EVERYTHING into the database as a project.

Besides, no matter what we do, it is probably just a matter of time before Free-DC rules the roost. LOL

IronBits 2009-05-03 03:53

I fully agree the data must go into the database, just bad timing if it's done during the time that Free-DC is closing the gap on the current stats system.

Cock-A-Doodle-Do :big grin:

MyDogBuster 2009-05-03 04:01

Maybe they can put off the load till after you kick our butts. I would vote for delaying the load.

Max - Gary. David has a valid point. They based their drive on the current stats setup so changing it wouldn't be kosher.

IronBits 2009-05-03 04:02

I was hoping for a better way to handle it - two stats systems perhaps
One for online (where almost all new users will be for some time)
One for offline (where the gurus know how to work the bothersome manual system, and get a speed advantage)

Brucifer 2009-05-03 04:05

How about just doing away with all team stats................. :smile:

mdettweiler 2009-05-03 06:15

Hmm...I see what you guys are saying. We had been planning to include all of the manual results in the stats for a long time hence, but you're right, this is probably a bad time to do it with all the Free-DCers counting on zonking ROLP in 31 days.

We'd considered having separate databases for manual and LLRnet, but that has a number of other issues. For instance, I'd always thought it would be really neat to have everything counted together in the "Drive Progress" scatter plot--if we included all results, manual and LLRnet, then it would be a better representation of the drives' progress.

Also, as Ian said, Free-DC has a lot of manual work under its belt, too--Carlos and Beyond both put in a heck of a lot of work manually. That would, most likely, somewhat balance out the increase that ROLP would get from including manual results in the stats.

Though, yes, I do agree that we should hold off for the time being, in light of the Free-DC project-of-the-month thing, so as not to confuse all those users who are watching the stats with an eagle eye yet aren't fully clued in to the existence of manual results. Gary, how would you feel about holding off until a week or so after Free-DC is done with their push? That way, it won't seem as much like the ROLP-affiliated admins are "cheating" at such a crucial time (which we're not, since it was planned for a while anyway, but as David said, it could look a lot like that if it was done right now).

Gary, I'll still download all the manual result files that you put up on crunchford as soon as I can--this will give me a chance to get a lot of the heavy lifting worked out so that as soon as we're ready to do the import, all I have to do is send the files to AMDave and we're all set.

Max :smile:

gd_barnes 2009-05-03 06:31

Wow, folks. This has been in the works for ages. Max has brought it up numerous times and we finally go to do it and now the objection comes? What's up with that?

Initially I objected because it will be a tremendous amount of work but Max has some automation set up for it and he has voluntered so we're going ahead with it.

Carlos, the big former Free-DC promoter here, was livid that we would NOT do it due to the large amount of manual work that he did. Therefore I'm frankly baffled why the objection now.

To make a project popular, we must have stats. They are the backbone of our project now. Therefore I propose the following:

1. Max, get all of the files properly created in LLRnet format with the correct UserID's, dates, etc. with the results that I have sent you. That alone will take a tremendous amount of time.

2. After you're done with #1, I'll send you more files for the 8th/9th/10th drives (much smaller manual work) as well as the individual-k drive. You can then hack away at those for an extended period.

3. If you are able to finish #'s 1 and 2 plus some additional straggling stuff that I'll send you like the mini-drive in < 3 months, then hold on to them until 3 total months has elapsed.

4. After 3 months; on August 1st, begin loading all of the files into the stats database.

David, by that time it's likely that Free-DC will be so far ahead that the manual loading of files won't matter. Keep in mind that as Max said, Carlos and several Free-DCers will have a large amount of manual files done from our 1st 3 drives.

Does that sound reasonable and politically correct to everyone?


gd_barnes 2009-05-03 06:39


In response to a couple of points in your post:

I apologize for the timing of this. We in no way intended to keep the carrot just out of reach. It's just the natural progression of the project. I hope that holding off on the manual stats import for 3 months will make it more palatable to people.

I had no idea that Free-DC had made NPLB the project of the month. That is VERY cool! A big thanks to you, AMDave, and Bok for your tireless efforts on the servers, stats, and promotion of the project at Free-DC!

Does my proposed solution seem like a fair one?


All times are UTC. The time now is 13:46.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.