Automatic Submission Spider for Workers...

[Xyzzy]

We tested the spider and it works well.

We will remove the factor found lines and deal (somehow) with those manually because PrimeNet shorted us, which we suppose is like playing the lottery. (See image attached.)

We have had great trouble in the past submitting our "mega-dumps" but we are lazy and that seemed like the only way to submit work without error. We will now use the spider, perhaps once a day.

Question: What happens if mfaktc writes to results.txt during the time the spider is working? Do we need to ensure that mfaktc is in the middle of a long run to be safe?

PS - We had to manually run "sudo apt-get install libwww-perl" to get the spider to work. This was not an easily figured out solution on our end because there are billions (?) of Perl packages. One would think a package like that would be installed by default.

Thanks for the script!

[chalsall]

Originally Posted by Xyzzy

We will remove the factor found lines and deal (somehow) with those manually because PrimeNet shorted us, which we suppose is like playing the lottery. (See image attached.)

Yes; this is a well documented "bug" with Primenet. George and James are aware of it, and hopefully it can be fixed sometime soon... (?)

Originally Posted by Xyzzy

We will now use the spider, perhaps once a day.

I, personally, use it twice an hour via a crontab entry:

9,39 * * * * ~/prime/gpu/tf/submit_spider ~/prime/gpu/tf/ >/dev/null

Originally Posted by Xyzzy

Question: What happens if mfaktc writes to results.txt during the time the spider is working? Do we need to ensure that mfaktc is in the middle of a long run to be safe?

Nope, you don't need to worry about this. The spider is intended to be (mostly) fully automatic. The code in the script:

# Move the results file so it won't get clobbered.
rename("results.txt","submitting_${Now}.txt");

# Sleep for two seconds to ensure any other users of results.txt finish up.
sleep(2);

... manages the situation where mfakt[c|o] and the spider happen to be working on the file at the same instant. It's known as a "reader - writer" relationship; only one program writes, only one reads. (To get "geeky", this leverages on the fact that a file can be renamed by one program, but if another program already has an open "file-handle" for said file any changes to it are preserved.)

Originally Posted by Xyzzy

PS - We had to manually run "sudo apt-get install libwww-perl" to get the spider to work. This was not an easily figured out solution on our end because there are billions (?) of Perl packages. One would think a package like that would be installed by default.

Interesting observation. What distro are you using? I'll make a note on the spider's documentation page.

[Xyzzy]

I, personally, use it twice an hour via a crontab entry:

How do you deal with factors found if it is automated? (Should the script be modified to pull out factor found lines and put them into a different file? Or maybe rewrite them back into results.txt?)

Interesting observation. What distro are you using? I'll make a note on the spider's documentation page.

Ubuntu 12.04 (LTS)

[chalsall]

Originally Posted by Xyzzy

How do you deal with factors found if it is automated? (Should the script be modified to pull out factor found lines and put them into a different file? Or maybe rewrite them back into results.txt?)

I, personally, don't particularly care about "proper" credits. And I would argue that George / Scott (and now James) should fix a bug which has been communally known for several years now, rather than having other programs and humans do additional work. Trivial to fix on Primenet (one regex test).

Originally Posted by Xyzzy

Ubuntu 12.04 (LTS)

Thanks for that information.

[Xyzzy]

The only reason we concern ourself with the factor thingie is because you have a report that shows the expected number of factors for the amount of work we have done.

http://www.gpu72.com/reports/worker/...4f765eb0f6ca8/

Since it is trivial to spoof results to gain false credit, we like to think that the expected factors found metric would allow others to see that our work is legit.

[chalsall]

Originally Posted by Xyzzy

Since it is trivial to spoof results to gain false credit, we like to think that the expected factors found metric would allow others to see that our work is legit.

GPU72 calculates this metric based on its own assignments. It doesn't matter if Primenet erroneously records the factor-found result as being found via P-1.

And, there will often be a large deviation of "Expected" vs "Found". Some are "lucky", some are "unlucky". If it's more than 10% out we get suspicious (and I sometimes waste my own GPU cycles double checking TFing results -- I have never found a cheater), but your particular results do not appear to be non-nominal.

[Xyzzy]

Well, the spider is now running automatically, and it works.



Now we have to find something else to "fix".

[James Heinrich]

Originally Posted by chalsall

Yes; this is a well documented "bug" with Primenet. George and James are aware of it, and hopefully it can be fixed sometime soon... (?)

I have been (still am) on vacation for the last long while. I'll be back in the frozen land I call home in about a week though, and I'll try and keep Primenet results submission fixing near the top of the huge pile of catch-up I have to sort through then.

[Bdot]

It appears submit_spider does not do well with the new primenet mechanism. Until I have time to fix the script I probably won't submit my results ...
at least not regularly - I hope there is still some page to manually submit things

[chalsall]

Originally Posted by Bdot

It appears submit_spider does not do well with the new primenet mechanism. Until I have time to fix the script I probably won't submit my results ...
at least not regularly - I hope there is still some page to manually submit things

Hmmmm... I'll take a look at this tomorrow afternoon. Have a deliverable for tomorrow morning.

But, yes, the "Manual Submission" page still exists and works.

[chalsall]

Originally Posted by chalsall

Hmmmm... I'll take a look at this tomorrow afternoon. Have a deliverable for tomorrow morning.

Thanks to Bdot (who got to the problem before I did), the Submission Spider for Linux has been updated to handle the POST log-in function of Primenet.

Those who still use this script please upgrade, and automatic submission of results will once again work.