Automatic Submission Spider for Workers... - Page 2

oswald · 2012-01-08, 04:55

OK, after a reboot, I still receive the error.
http://www.mersenne.org/results/ did not show a result for M45833773 until I submitted it manually.

Now it shows 83.4765 Ghz-Days for no factor for M45833773 from 2^74 to 2^75 [mfaktc 0.18 barrett79_mul32].

submit_spider was able to handle M45833773 to 2^74. Maybe it doesn't like 2^75.

Any suggestions? I'll stay at 2^74 or lower for now.

Roy

Dubslow · 2012-01-08, 05:01

I would like to report that slightly over an hour ago, Prime95 was not able to contact PrimeNet: CURL reported a time out (waited 180 seconds). It should try again in the next ten minutes.

Edit: By which I mean it did actually work about 5 minutes before this original post.

oswald · 2012-01-08, 05:16

False alarm. It works now. I didn't expect that error message when it was the server instead.

Dubslow Thanks!

chalsall · 2012-01-08, 05:19

Quote:

Originally Posted by oswald

Code:

20120108_032058 INFO:  Successfully logged into PrimeNet.
Use of uninitialized value $GHzDays in concatenation (.) or string at /usr/local/bin/submit_spider line 151, <IN> line 1.
20120108_032121 INFO:  M49831559 submitted;  GHz Days credit.
20120108_032122 INFO:  Submission complete.  Moving results file to "submitted_20120108_032058.txt"

Thanks for that information. Very important.

You've found a bug in my code -- it doesn't properly handle the situation where the spider can log into PrimeNet, but then doesn't properly submit the result.

I will look into this when I'm not suffering the effects of a wedding...

Quote:

Originally Posted by Dubslow

I would like to report that slightly over an hour ago, Prime95 was not able to contact PrimeNet: CURL reported a time out (waited 180 seconds). It should try again in the next ten minutes.

Also useful empirical data.

kladner · 2012-01-08, 05:27

Best wishes for the wedding, and the recovery therefrom!

oswald · 2012-01-08, 06:17

Quote:

Originally Posted by chalsall

I will look into this when I'm not suffering the effects of a wedding...

Happy to help. Love your program.

Linda tells me we are married, but I don't remember my wedding

oswald · 2012-01-08, 06:36

I'm thinking, this might be why I still had old LLTFs assigned to test and I was sure I had already tested them. I did have it deleting my old results.txt and it is possible they were tested but not successfully reported.

If that is the case, it is ok, I can test them again, I was just afraid I was losing my mind.

Dubslow · 2012-02-29, 19:14

Hey chalsall,
Regarding the submission spider, could you modify it to append submitted results to an already existing file, as opposed to creating a new one? So

Code:

else {
      Log(1, "Submission complete.  Moving results file to \"submitted_${Now}.txt\"");
      rename("submitting_${Now}.txt", "submitted_${NOW}.txt");
   }

would become something like

Code:

else {
      Log(1, "Submission complete.  Moving results file to \"submitted_${Now}.txt\"");
      append("submitting_${Now}.txt", "submitted.txt");
   }

I tried Googling it, but the page I found says you have to modify how you open the file, and that would substantially complicate the code; if that's true, is it possible to run another script/prog located in the $PWD, as in instead of the above you'd have

Code:

else {
      Log(1, "Submission complete.  Moving results file to \"submitted_${Now}.txt\"");
      run("append.sh submitting_${NOW}.txt");
   }

where you call append.sh with "submitting..." as the first argument? If that's possible I can fill in the rest myself. Edit: Just Googled that, and that is possible, though of course it would be simpler to have it done inside the perl. ~~I will implement it with bash until you render judgement.~~

Thanks of course for making the rest of the Spider.
Edit2: Whoops, hehe. If I can call a bash script, why not just call cat?

Code:

else {
      Log(1, "Submission complete.  Appending results file to \"submitted.txt\"");
      system("cat submitting_${Now}.txt >> submitted.txt");             
   }

Do you see any reason why this wouldn't work? Edit3: Need to delete submitting_Now.txt

Source: http://perlmeme.org/faqs/system/system.html

Dubslow · 2012-03-01, 03:55

Final modification:

Code:

else {
      Log(1, "Submission complete.  Appending results file to \"submitted.txt\"");
      system("cat submitting_${Now}.txt >> submitted.txt"); # Not portable.
      system("rm submitting_${Now}.txt");                   # Ditto.
   }

I do this so that I can later upload results to Mersenne-aries.

Now, on a more-or-less unrelated topic: chalsall, for the GPU272 spiders, do you communicate with PrimeNet via the proper API, or just a hack of the manual testing pages, as in this particular spider? I'm inclined to think it's the former, based on the talk of CompIDs and assignment progress reports.

...The reason I ask is that if the submission_spider were to go through the API, it'd save PrimeNet a lot of work generating HTML pages for manual results. However there are a number of problems with this.

http://v5.mersenne.org/v5design/v5webAPI_0.97.html#uc

Code:

The 'uc' CPU/app info transaction may be sent as often as the client program wants, but it MUST be sent at least once before any assignment requests, updates or results...

That includes

Code:

g - program's self-assigned permanent ID (guid) --problem
hg - machine's hardware hash ID (guid) --problem
wg - machine's Windows hardware hash ID (guid) --problem
a - application version string (min length 10, max length 64) --less-of-a-problem
Version string MUST be formatted in "platform, code, version, build" style,  where the broadest category is leftmost and the most specific or varied rightmost, etc., however specific CPU identifier strings should be coordinated among client program developers to decide specific content conventions.
The following examples could be considered valid, well-formed versions:
"Windows, Prime95, v24.1, build 1"
"Windows, Prime95, v24.2, build 2"
"Linux, MPrime, v24.2, build 1"
"Mac, GLucas, v13.2, build 9"
"Windows, GridAppX, v1.2, build 9"
"Windows, Prime95, v25.0, test 1/15/2006", etc.
c - CPU model string (min length 8, max length 64) --problem
Well-formed, uniquely identified CPU model strings MUST be formatted in "manufacturer, model" style, where the broadest category is leftmost and the most specific or varied rightmost, etc., however specific CPU model identifier strings should be coordinated among client program developers to decide a specific content convention.
The following examples could be considered valid, well-formed CPU model strings:
"Intel, P4-M"
"Intel(R) Pentium(R) 4"
"Intel(R) Celeron(R)"
"Intel Pentium Pro"
"AMD Athlon(tm) XP"
f - CPU features string (min length 0, max length 64) --problem(can be null?)
Well-formed, comma-separated CPU feature strings should be coordinated among client program developers to decide a specific content convention.
The following example could be considered valid, well-formed CPU features string:
"RDTSC,CMOV,PREFETCH,MMX,SSE,SSE2"
L1 - level 1 cache of CPU in KB (integer; set 0 if unavailable)
L2 - level 2 cache of CPU in KB (integer; set 0 if unavailable)
L3 - level 3 cache of CPU in KB (integer; optional)
np - number of physical CPUs/cores available to run assignments (integer >= 1)
hp - number of hyperthreaded CPUs on each physical CPU (integer >= 0)
m - number of megabytes of physical memory (integer >= 0; set 0 if unavailable)
s - speed of CPU in Mhz; assumes all CPUs are same speed (integer)
h - hours per day CPU runs application (integer 0-24)
r - rolling average (integer; set 0 if unavailable)
u - existing server account userID to bind CPU's owning user (max length 20; may be null, see notes)

Along with those problems, the following also apply: http://v5.mersenne.org/v5design/v5webAPI_0.97.html#ar

Code:

k - assignment key (guid; must match k given in 'ra' or 'ga' transaction response, or 0)
...
When assignment key k=0, no credit is granted. This is called the PrimeNet 'Unexpected Results' Policy.

It is possible for these things to be worked around, however it would mean no small amount of work on George's part, and would duplicate more than a bit of the work Christenson is doing to automate mfaktc (and I suddenly see why that is so much harder than it sounds).

chalsall · 2012-03-01, 15:02

Quote:

Originally Posted by Dubslow

I do this so that I can later upload results to Mersenne-aries.

A good idea. I'll incorporate it into version 0.3 of the spider. But note the second call to the "system", the "rm" function, can be done natively in Perl with the "unlink()" function used elsewhere in the spider.

Quote:

Originally Posted by Dubslow

Now, on a more-or-less unrelated topic: chalsall, for the GPU272 spiders, do you communicate with PrimeNet via the proper API, or just a hack of the manual testing pages, as in this particular spider? I'm inclined to think it's the former, based on the talk of CompIDs and assignment progress reports.

Spidy uses various techniques to get assignments and state knowledge. For the getting and returning of assignments, it uses the client/server API.

Dubslow · 2012-03-01, 18:35

Quote:

Originally Posted by chalsall

A good idea. I'll incorporate it into version 0.3 of the spider. But note the second call to the "system", the "rm" function, can be done natively in Perl with the "unlink()" function used elsewhere in the spider.

I had seen that, but the name made me think there was something more to it and that I shouldn't mess with it. (What happened to 0.2?)

2012-01-08, 05:01	#13
Dubslow Basketry That Evening! "Bunslow the Bold" Jun 2011 40<A<43 -89<O<-88 1C35₁₆ Posts	I would like to report that slightly over an hour ago, Prime95 was not able to contact PrimeNet: CURL reported a time out (waited 180 seconds). It should try again in the next ten minutes. Edit: By which I mean it did actually work about 5 minutes before this original post. Last fiddled with by Dubslow on 2012-01-08 at 05:10

2012-02-29, 19:14	#19
Dubslow Basketry That Evening! "Bunslow the Bold" Jun 2011 40<A<43 -89<O<-88 3·29·83 Posts	Hey chalsall, Regarding the submission spider, could you modify it to append submitted results to an already existing file, as opposed to creating a new one? So Code: else { Log(1, "Submission complete. Moving results file to \"submitted_${Now}.txt\""); rename("submitting_${Now}.txt", "submitted_${NOW}.txt"); } would become something like Code: else { Log(1, "Submission complete. Moving results file to \"submitted_${Now}.txt\""); append("submitting_${Now}.txt", "submitted.txt"); } I tried Googling it, but the page I found says you have to modify how you open the file, and that would substantially complicate the code; if that's true, is it possible to run another script/prog located in the $PWD, as in instead of the above you'd have Code: else { Log(1, "Submission complete. Moving results file to \"submitted_${Now}.txt\""); run("append.sh submitting_${NOW}.txt"); } where you call append.sh with "submitting..." as the first argument? If that's possible I can fill in the rest myself. Edit: Just Googled that, and that is possible, though of course it would be simpler to have it done inside the perl. ~~I will implement it with bash until you render judgement.~~ Thanks of course for making the rest of the Spider. Edit2: Whoops, hehe. If I can call a bash script, why not just call cat? Code: else { Log(1, "Submission complete. Appending results file to \"submitted.txt\""); system("cat submitting_${Now}.txt >> submitted.txt"); } Do you see any reason why this wouldn't work? Edit3: Need to delete submitting_Now.txt Source: http://perlmeme.org/faqs/system/system.html Last fiddled with by Dubslow on 2012-02-29 at 19:55

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
GPU to 72 spider poaching exponents?	ixfd64	GPU to 72	2	2014-07-27 20:38
It Seems The Great Spider ...	Dubslow	Forum Feedback	13	2012-06-01 13:39
Automatic temperature control	TObject	Software	13	2012-04-27 19:33
Went from 8 workers to 4 workers on v26.6 upgrade	dmoran	Software	13	2011-05-23 12:36
Manual submission of automatic assignment result	tichy	PrimeNet	4	2010-12-17 09:57

2012-01-08, 04:55	#12
oswald Apr 2011 in vivo 3×5² Posts	OK, after a reboot, I still receive the error. http://www.mersenne.org/results/ did not show a result for M45833773 until I submitted it manually. Now it shows 83.4765 Ghz-Days for no factor for M45833773 from 2^74 to 2^75 [mfaktc 0.18 barrett79_mul32]. submit_spider was able to handle M45833773 to 2^74. Maybe it doesn't like 2^75. Any suggestions? I'll stay at 2^74 or lower for now. Roy

2012-01-08, 05:16	#14
oswald Apr 2011 in vivo 75₁₀ Posts	False alarm. It works now. I didn't expect that error message when it was the server instead. Dubslow Thanks!

2012-01-08, 05:27	#16
kladner "Kieren" Jul 2011 In My Own Galaxy! 23656₈ Posts	Best wishes for the wedding, and the recovery therefrom!

2012-01-08, 06:36	#18
oswald Apr 2011 in vivo 4B₁₆ Posts	I'm thinking, this might be why I still had old LLTFs assigned to test and I was sure I had already tested them. I did have it deleting my old results.txt and it is possible they were tested but not successfully reported. If that is the case, it is ok, I can test them again, I was just afraid I was losing my mind.