mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > No Prime Left Behind

Reply
 
Thread Tools
Old 2010-04-16, 16:31   #1
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3·2,083 Posts
Default Loading of manual results into the DB

Admin edit: split this conversation off into a separate thread due to its having veered off somewhat significantly from the general theme of the "News" thread

Drive #3 has now been imported into the DB. More should follow soon!

Last fiddled with by mdettweiler on 2010-04-17 at 10:53
mdettweiler is offline   Reply With Quote
Old 2010-04-16, 22:25   #2
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

23×349 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
Drive #3 has now been imported into the DB.
So you have to redo this!

All LLRnet-results (not sure yet, but it seems so) were added under the contributor "Unknown"!
See the Top Participants.

Rank Participant Team Pairs Returned Pair Score
11 Unknown - 399398 584,472.485

The first range from Drive #3 was at n=340k and the first prime found there

Code:
user=kar_bon
[03/01/08 15:52:33]
399*2^341641-1 is prime!  Time : 150.0 sec.
but the primelist for filter "k=399" here says

Code:
Participant Date Time k Base n Sign One Seconds Server Port Pair Score Prime Score Digits 
Bruce 2009-11-29 15:09:08 2399 2 224852 - 1 1475 GB 7000 0.315990 0.177628 67691 
Bruce 2009-11-29 14:08:13 2399 2 52196 - 1 133 GB 7000 0.017028 0.002222 15716 
PCZ 2009-01-13 12:52:53 1399 2 142227 - 1 181 IB 9000 0.126428 0.044954 42818 
gd_barnes 2008-05-24 06:22:05 399 2 496100 - 1 589 IB 5000 1.538220 1.907777 149344 
Unknown 2008-04-10 13:03:00 399 2 341641 - 1 50 MN 0003 0.729491 0.623060 102847 
Beyond 2008-03-20 07:25:00 399 2 290555 - 1 836 MN 0003 0.527639 0.383270 87469
Sorry guys! Seems you overwrote all LLRnet-results with 'Unknown'!

'Unknown' owns 18 primes now!

PS: As I can say, Drive #3 got 386082 pairs done by LLRnet-server I5000/I8000.
The count (399398) from above is more than this! Question: Which pairs were false inserted, too?

Last fiddled with by kar_bon on 2010-04-16 at 22:41
kar_bon is online now   Reply With Quote
Old 2010-04-16, 22:39   #3
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

186916 Posts
Default

Quote:
Originally Posted by kar_bon View Post
So you have to redo this!

All LLRnet-results (not sure yet, but it seems so) were added under the contributor "Unknown"!
See the Top Participants.

Rank Participant Team Pairs Returned Pair Score
11 Unknown - 399398 584,472.485

The first range from Drive #3 was at n=340k and the first prime found there

Code:
user=kar_bon
[03/01/08 15:52:33]
399*2^341641-1 is prime!  Time : 150.0 sec.
but the primelist for filter "k=399" here says

Code:
Participant Date Time k Base n Sign One Seconds Server Port Pair Score Prime Score Digits 
Bruce 2009-11-29 15:09:08 2399 2 224852 - 1 1475 GB 7000 0.315990 0.177628 67691 
Bruce 2009-11-29 14:08:13 2399 2 52196 - 1 133 GB 7000 0.017028 0.002222 15716 
PCZ 2009-01-13 12:52:53 1399 2 142227 - 1 181 IB 9000 0.126428 0.044954 42818 
gd_barnes 2008-05-24 06:22:05 399 2 496100 - 1 589 IB 5000 1.538220 1.907777 149344 
Unknown 2008-04-10 13:03:00 399 2 341641 - 1 50 MN 0003 0.729491 0.623060 102847 
Beyond 2008-03-20 07:25:00 399 2 290555 - 1 836 MN 0003 0.527639 0.383270 87469
Sorry guys! Seems you overwrote all LLRnet-results with 'Unknown'!

'Unknown' owns 18 primes now!
Some of the early LLRnet results from the C*** servers never got imported into the DB; unless you still have the original files sitting around, the data on who specifically did what in those ranges has long since been lost. Those have therefore been imported as "Unknown".

However, if you do have the original results for those, please send them to me--I'll then send them to Dave and ask him to do a special re-import on them to overwrite the "Unknown"'s.
mdettweiler is offline   Reply With Quote
Old 2010-04-16, 22:51   #4
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

23·349 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
Some of the early LLRnet results from the C*** servers never got imported into the DB; unless you still have the original files sitting around, the data on who specifically did what in those ranges has long since been lost. Those have therefore been imported as "Unknown".

However, if you do have the original results for those, please send them to me--I'll then send them to Dave and ask him to do a special re-import on them to overwrite the "Unknown"'s.
As mentioned those results were from IB5000 and IB 8000 servers!

Tell me which results you missing and i see what i got.
kar_bon is online now   Reply With Quote
Old 2010-04-17, 00:13   #5
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

624910 Posts
Default

Quote:
Originally Posted by kar_bon View Post
As mentioned those results were from IB5000 and IB 8000 servers!

Tell me which results you missing and i see what i got.
Okay, I've uploaded a dump of all results with user "Unknown" to:
http://www.noprimeleftbehind.net/unknown_results.txt Edit by Max: link taken offline (no longer needed)

A few of the "Unknown" results are just occasional pairs that you, myself, or Gary filled in during processing, but there definitely appear to be some whole ranges in there. They are:

1st Drive, n=510K-510K.5K
Strange...that range was done manually by me. It's possible that this range was accidentally mislabeled in manual-import preprocessing. This can be easily remedied with a quick SQL command.

1st Drive, n=543038-543749
This range was originally done by the C443 server. It looks like we missed one file when we manually downloaded the results from Carlos's web site and sent them to Dave for importing.

3rd Drive, n=340K-383274
Originally done by IB5000. This may have been before we moved the DB over to David's server; if it was still on Adam's machine, then we would have been doing these manually. It seems these files here were somehow neglected.

Karsten, if you have the original LLRnet-format results for the latter two, send them my way--I'll talk to Dave about figuring out some way to have the DB do a special import on them where it will use them to replace "Unknown" results where appropriate. As for the first one (the 1st Drive range done by me), since that was a manual range done entirely by one person it's simply a matter of mislabeling and can be easily corrected.

BTW, I didn't see any "Unknown" results from IB8000 ranges; could you possibly point those out to me?

Last fiddled with by mdettweiler on 2010-05-21 at 18:39
mdettweiler is offline   Reply With Quote
Old 2010-04-17, 00:46   #6
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

23×349 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
Karsten, if you have the original LLRnet-format results for the latter two, send them my way
Mailed!

Quote:
Originally Posted by mdettweiler View Post
BTW, I didn't see any "Unknown" results from IB8000 ranges; could you possibly point those out to me?
IB8000 seems ok, as I can say from here.

But please check Drive #3 IB5000 again:
For example this primes:
Unknown 2008-11-10 10:24:00 371 2 571042

but it was found by Gary!

From resultfile "results_20081104_0657_IB_nplb_5000.txt":
Code:
user=gd_barnes
[11/04/08 05:35:56]
371*2^571042-1 is prime!  Time :  sec.
There're several ranges done by IB5000 on Drive #3, see here: I've documented all ranges with server or contributor there.
kar_bon is online now   Reply With Quote
Old 2010-04-17, 03:37   #7
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3·2,083 Posts
Default

Quote:
Originally Posted by kar_bon View Post
Mailed!
Thanks, got 'em.
Quote:
IB8000 seems ok, as I can say from here.

But please check Drive #3 IB5000 again:
For example this primes:
Unknown 2008-11-10 10:24:00 371 2 571042

but it was found by Gary!

From resultfile "results_20081104_0657_IB_nplb_5000.txt":
Code:
user=gd_barnes
[11/04/08 05:35:56]
371*2^571042-1 is prime!  Time :  sec.
There're several ranges done by IB5000 on Drive #3, see here: I've documented all ranges with server or contributor there.
Now that is strange. No results for that n appear in the list of "Unknown" results I posted earlier, yet I generated that list with the SQL command "select * from results where username = 'Unknown'", which should produce each and every "Unknown" result in the DB. Something isn't adding up here.

To investigate this further I tried selecting just this one k/n pair from the DB:
Code:
mysql> select * from results where n = 571042 and k = 371;
+-----------+------------+----------+-----+------+--------+------+-----+-------+----------+---------+--------+------+----------------+-------------+
| username  | date       | time     | k   | base | n      | sign | one | prime | residual | seconds | server | port | pair_score     | prime_score |
+-----------+------------+----------+-----+------+--------+------+-----+-------+----------+---------+--------+------+----------------+-------------+
| gd_barnes | 2008-11-04 | 05:35:56 | 371 |    2 | 571042 | -    |   1 |     0 |          |       0 | IB     | 5000 | 2.038056036025 |        NULL |
+-----------+------------+----------+-----+------+--------+------+-----+-------+----------+---------+--------+------+----------------+-------------+
1 row in set (0.13 sec)
So it is in there under Gary's name, not Unknown. One thing I did find strange is that the "prime" column seemed to be set to 0 despite the number being prime but it's possible I'm just misinterpreting Dave's method of filing primes in the DB.

Yet the website lists it under "Unknown".

Dave, any idea of the cause of this apparent disconnect?
mdettweiler is offline   Reply With Quote
Old 2010-04-17, 04:52   #8
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

2×3×7×241 Posts
Default

Why are we making this so extremely difficult? I have now stepped in and recommended that we dispense with this entire reloading of every result that we have into the DB.

The intent was that this be a manual results import not an all results import, which would take months.

If some stuff got overwritten, that is very bad news and shows that we did not properly analyze the situation ahead of time. If that is what happened, is there a way that we can restore them?

There should be very few "unknown" results. Perhaps no more than 50-100 on the entire project. For manual results, we should know who did all of them and it's just a matter of looking it up in the 1st post of each drive. The only ones that should end up entered in the DB as "unknown" are where the server might have somehow missed 1-2 of them and either Max, Karsten, or I ran the pairs manually and put them in our file that we keep so that they matched up with original sieve file.

Max, all results for the entire 5th thru 10th drives and mini-drive are now on Jeepford. Please analyze which ones of them were done manually, associate who did them and when they were done, and load only those into the DB. For each drive, the manual loading should go fairly quickly. Since this project started the 5th drive, 95-98% of all results have been done by the servers. The lion's share of the manual stats import is coming from the 1st thru 3rd drives but even those were largely done by servers.

Going forward, the only drives that should take a little while to load into the DB are the fully manual ones; that is the individual-k and mini drive. Everything else after the 3rd drive should go very fast.

In the future, before loading any manual pairs into the DB, I want to review what is being loaded.


Gary

Last fiddled with by gd_barnes on 2010-04-17 at 05:12
gd_barnes is offline   Reply With Quote
Old 2010-04-17, 08:51   #9
AMDave
 
AMDave's Avatar
 
Jan 2006
deep in a while-loop

2·7·47 Posts
Default

grep search shows the following result

results_20081104_0657_IB_nplb_5000.txt:
user=gd_barnes^M
[11/04/08 05:35:56]^M
371*2^571042-1 prime! Time : sec.^M

results_20100415_2035_MN_nplb_0003.txt:
user=Unknown
[2008-11-10 10:24:00]
371*2^571042-1 is prime! Time : 0.0 sec.

Re the first file, that whole file is full of CTRL characters.
I will check if that has had an impact on the loading.
The prime detection is looking for "prime!" so the first instance should have been loaded and registered as a prime.
It could be the missing search-time. I will test the parsing.

Re the second file, this pair was not loaded because the pair already exists in the results table.
Why the pair was subsequently submitted again in manual search I do not know.

Last fiddled with by AMDave on 2010-04-17 at 09:00
AMDave is offline   Reply With Quote
Old 2010-04-17, 08:52   #10
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

53508 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
So it is in there under Gary's name, not Unknown. One thing I did find strange is that the "prime" column seemed to be set to 0 despite the number being prime but it's possible I'm just misinterpreting Dave's method of filing primes in the DB.

Yet the website lists it under "Unknown".
As i can say from an export of Adam's DB from 2008-09-15 17:09:22 there are tables for pairs and an extra table for primes!

So the pair is correctly in that table you requested but the prime as shown here is from the prime-table and there is contributor = 'Unknown'!
kar_bon is online now   Reply With Quote
Old 2010-04-17, 09:26   #11
AMDave
 
AMDave's Avatar
 
Jan 2006
deep in a while-loop

2·7·47 Posts
Default

suggest we splice these posts out of the News thread
AMDave is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
GPU id/name for manual results preda GPU Computing 15 2017-08-16 17:34
mersenne.ca and manual results Gordon PrimeNet 3 2015-08-31 03:08
manual results ramgeis PrimeNet 8 2013-05-30 06:33
manual results reporting bug for P-1 factoring dgould Information & Answers 4 2009-12-11 03:26
Manual Checkin of P-1 Results Unregistered PrimeNet 1 2004-05-18 03:15

All times are UTC. The time now is 00:38.

Mon May 25 00:38:27 UTC 2020 up 60 days, 22:11, 0 users, load averages: 1.46, 1.41, 1.39

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.