mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > Conjectures 'R Us

Reply
 
Thread Tools
Old 2010-01-03, 19:44   #23
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

2×179 Posts
Default

I am running prpnet server on Win 2003 32bit.
Have emailed you my data directory and the pfgw files.
PCZ is offline   Reply With Quote
Old 2010-01-03, 23:15   #24
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

16616 Posts
Thumbs up

Used prpadmin to remove candidates with a factors file, worked fine.
Also added more pairs of a different base, that went well.

Did find a few more of the duplicate errors though.

See attached log.txt
You wil see two successful candidate removals by factor file and addition of 14.000 more pairs.
Server didn't choke up at all, which v2 would have.

Thumbs up from me
Ver 2x would have been on it's knees crying if asked to do what i am doing with V3.


Mark
What would you say a sensible limit is for number of candidates to load in one go / total.
On a seperate instance i did try to upload 650,000 candidates but it didn't handle that at all well.
Took ages and the DB refused any more connections with error code=1040.

Last fiddled with by PCZ on 2010-01-03 at 23:18
PCZ is offline   Reply With Quote
Old 2010-01-04, 00:27   #25
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

5,779 Posts
Default

Quote:
Originally Posted by PCZ View Post
Used prpadmin to remove candidates with a factors file, worked fine.
Also added more pairs of a different base, that went well.

Did find a few more of the duplicate errors though.

See attached Attachment 4521
You wil see two successful candidate removals by factor file and addition of 14.000 more pairs.
Server didn't choke up at all, which v2 would have.

Thumbs up from me
Ver 2x would have been on it's knees crying if asked to do what i am doing with V3.


Mark
What would you say a sensible limit is for number of candidates to load in one go / total.
On a seperate instance i did try to upload 650,000 candidates but it didn't handle that at all well.
Took ages and the DB refused any more connections with error code=1040.
Did the candidates that were marked as duplicates get loaded? Are there any missing? I need to know if this issue with duplicates is causing the server to not load some candidates.

The problem with 650,000 is that the server computes the decimal length of each number, but only does so after all numbers are received. I suspect that MySQL ran into issues when trying to open that cursor. Did the server eventually come back to life so that clients could connect? I wouldn't put a hard and fast limit on the file size for three reasons. First, I don't read the entire file to count the number of entries. Second, I don't know what a practical limit would be. Third, I can probably address this through some creative coding in the server.
rogue is offline   Reply With Quote
Old 2010-01-04, 00:40   #26
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

1011001102 Posts
Default

What can i use to look at the tables ?
Something like the GUI with MS SQL.

Edit
Found some GUI tools.
Exported the candidates table as a csv
http://itsoapbox.com/candidates.zip
http://itsoapbox.com/pfgw.zip

Last fiddled with by PCZ on 2010-01-04 at 01:20
PCZ is offline   Reply With Quote
Old 2010-01-04, 01:25   #27
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

577910 Posts
Default

Quote:
Originally Posted by PCZ View Post
What can i use to look at the tables ?
Something like the GUI with MS SQL.

Edit
Found some GUI tools.
Exported the candidates table as a csv
http://itsoapbox.com/candidates.zip
http://itsoapbox.com/pfgw.zip
Could you please answer my questions?
rogue is offline   Reply With Quote
Old 2010-01-04, 01:35   #28
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

16616 Posts
Default

When i loaded up a large amount of pairs the DB connection was lost.
I restarted sql but the server was very sluggish.
SQL didn't recover itself, i restarted it.
Between sql and prpnet they were hogging the hardrive.
I left it alone for 30mins to see if it would eventually settle down but it didn't.



The candidates get loaded into the table.
Candidates get handed out to a client.
Sometimes a line or 2 down in the log will show duplicate error.
Candidate gets returned and and gets added to completed tests log.

Only candidate missing is one of the ten base 2 numbers i added.

I will add those few base 2 numbers again and see if problem repeats itself.
Tried adding those pairs again but i couldn't.

Enter option from menu: 4
Enter ABC file to be imported: sr_2.pfgw
ERR: 2949*2^412111-1 is a duplicate and has been rejected
ERR: 2941*2^416993-1 is a duplicate and has been rejected
ERR: 2945*2^436732-1 is a duplicate and has been rejected
ERR: 2943*2^448416-1 is a duplicate and has been rejected
ERR: 2933*2^466692-1 is a duplicate and has been rejected
ERR: 2949*2^500481-1 is a duplicate and has been rejected
ERR: 2941*2^526493-1 is a duplicate and has been rejected
ERR: 2945*2^546922-1 is a duplicate and has been rejected
ERR: 2937*2^555062-1 is a duplicate and has been rejected
ERR: 2931*2^584103-1 is a duplicate and has been rejected
Total candidates received = 10
Failed Inserts = 0
Not Inserted = 0
New candidates added = 0
Duplicates rejected = 10
Bad entries from ABC file = 0

Last fiddled with by PCZ on 2010-01-04 at 02:00
PCZ is offline   Reply With Quote
Old 2010-01-04, 02:14   #29
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

5,779 Posts
Default

[QUOTE=PCZ;200790]
Enter option from menu: 4
Enter ABC file to be imported: sr_2.pfgw
ERR: 2949*2^412111-1 is a duplicate and has been rejected
ERR: 2941*2^416993-1 is a duplicate and has been rejected
ERR: 2945*2^436732-1 is a duplicate and has been rejected
ERR: 2943*2^448416-1 is a duplicate and has been rejected
ERR: 2933*2^466692-1 is a duplicate and has been rejected
ERR: 2949*2^500481-1 is a duplicate and has been rejected
ERR: 2941*2^526493-1 is a duplicate and has been rejected
ERR: 2945*2^546922-1 is a duplicate and has been rejected
ERR: 2937*2^555062-1 is a duplicate and has been rejected
ERR: 2931*2^584103-1 is a duplicate and has been rejected/QUOTE]

Which of these didn't get sent to a client? Can you do "select * from Candidate where CandidateName = '<xx>';" and send the output? Is there an entry in the CandidateTest table for it?

I think that I have narrowed down the issue with that large file, sr_3.pfgw. It seems that the socket on the server is full and cannot receive any more data while it is inserting candidates. For example, it reads from the socket and gets 10,000 or so, then while it is processing those 10,000 the admin tool is sending more (remember that these are asynchronous at the moment). The admin tool is flooding the socket on the server side and eventually is unable to send more data. The flooding of the socket on the server side is also preventing other clients from connecting. I have a couple of ideas for addressing this.

As for the sluggishness of your computer, it is more likely to be an issue with MySQL than PRPNet. As I am also new to MySQL, I don't know enough to determine how one can optimize performance on large tables.

If anyone else has experience with MySQL performance tuning, could you please assist?
rogue is offline   Reply With Quote
Old 2010-01-04, 02:56   #30
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

2·179 Posts
Default

So it should have been sent as it is smaller than all the base 3 candidates but for some reason it didn't.
It made it into the DB OK.

As to the sluggishness i did a bit of my own tuning.
Re installed mysql to a seperate hardrive and made sure the data was also on this HD.
Chose custom and made sensible choices during installation.
Having the database in c:\users\x wasnt good when it was building tables.

mysql> select * from Candidate where CandidateName = '2941*2^526493-1';
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| CandidateName | DecimalLength | CompletedTests | HasPendingTest | DoubleChec
ked | k | b | n | c | IsPRP | IsPrime | IsTwinPRP | IsTwinPrime |
SearchedForGFNDivisors | HasSierpinskiRieselPrime | LastUpdateTime |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| 2941*2^526493-1 | 158494 | 0 | 0 |
0 | 2941 | 2 | 526493 | -1 | 0 | 0 | 0 | 0 |
0 | 1 | 1262533329 |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
1 row in set (0.00 sec)

mysql>

2941*2^526493-1 never got sent to a client



One that did
mysql> select * from Candidate where CandidateName = '2933*2^466692-1';
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| CandidateName | DecimalLength | CompletedTests | HasPendingTest | DoubleChec
ked | k | b | n | c | IsPRP | IsPrime | IsTwinPRP | IsTwinPrime |
SearchedForGFNDivisors | HasSierpinskiRieselPrime | LastUpdateTime |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| 2933*2^466692-1 | 140492 | 1 | 0 |
0 | 2933 | 2 | 466692 | -1 | 0 | 1 | 0 | 0 |
0 | 1 | 1262533996 |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
1 row in set (0.00 sec)

mysql

Last fiddled with by PCZ on 2010-01-04 at 03:21
PCZ is offline   Reply With Quote
Old 2010-01-04, 13:37   #31
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

5,779 Posts
Default

Quote:
Originally Posted by PCZ View Post
So it should have been sent as it is smaller than all the base 3 candidates but for some reason it didn't.
It made it into the DB OK.

As to the sluggishness i did a bit of my own tuning.
Re installed mysql to a seperate hardrive and made sure the data was also on this HD.
Chose custom and made sensible choices during installation.
Having the database in c:\users\x wasnt good when it was building tables.

mysql> select * from Candidate where CandidateName = '2941*2^526493-1';
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| CandidateName | DecimalLength | CompletedTests | HasPendingTest | DoubleChec
ked | k | b | n | c | IsPRP | IsPrime | IsTwinPRP | IsTwinPrime |
SearchedForGFNDivisors | HasSierpinskiRieselPrime | LastUpdateTime |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
| 2941*2^526493-1 | 158494 | 0 | 0 |
0 | 2941 | 2 | 526493 | -1 | 0 | 0 | 0 | 0 |
0 | 1 | 1262533329 |
+-----------------+---------------+----------------+----------------+-----------
----+------+------+--------+------+-------+---------+-----------+-------------+-
-----------------------+--------------------------+----------------+
1 row in set (0.00 sec)l
I now see the reason, which is good because it tells me that PRPNet is working as it should. HasSierpinskiRieselPrime=1. Since this is a Sierpinski/Riesel type server, once a prime is found for a k/b/c combination, no other tests for that k/b/c combination will be sent out. Had the servertype been different, such as FixedKBNC, this would have been sent out.

As for your performance issue, has it been resolved?
rogue is offline   Reply With Quote
Old 2010-01-04, 13:54   #32
PCZ
 
PCZ's Avatar
 
Jun 2006
Chertsey Surrey UK

16616 Posts
Default

Type 1 is definitely not what i want.
All pairs should be processed, smallest first.
One of the fixed options then, ill change the ini and restart.

Will this change the tables though ?
Now the pairs are loaded will they stay at type 1 ?

Performance is fine now.

Last fiddled with by PCZ on 2010-01-04 at 14:50
PCZ is offline   Reply With Quote
Old 2010-01-04, 15:36   #33
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

10110100100112 Posts
Default

Quote:
Originally Posted by PCZ View Post
Type 1 is definitely not what i want.
All pairs should be processed, smallest first.
One of the fixed options then, ill change the ini and restart.

Will this change the tables though ?
Now the pairs are loaded will they stay at type 1 ?

Performance is fine now.
You want to set that to 0 in MySQL.

update Candidate set HasSierpinskiRiesel = 0 where HasSierpinskiRiesel = 1;
rogue is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
PRPNet 5.4.3 Released rogue Software 152 2020-03-30 17:01
PSP goes prpnet ltd Prime Sierpinski Project 86 2012-06-06 02:30
PRPNet 4.0.0 Released rogue Software 84 2011-11-16 21:20
PRPNet 4.0.1 Released Joe O Sierpinski/Riesel Base 5 1 2010-10-22 20:11
PRPNet released! rogue Conjectures 'R Us 250 2009-12-27 21:29

All times are UTC. The time now is 07:04.

Thu Jul 2 07:04:57 UTC 2020 up 99 days, 4:38, 0 users, load averages: 1.44, 1.51, 1.33

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.