Automatic fetch of Trial Factoring work for GPU mfakt*

[LaurV]

Ok, now the fetching of P95 work is solved. I need a toy to fetch TF assignments from GPU272. In fact, what I need the toy/spider/minion/terrorist to do is:

1. read a command (or ini, cfg, etc) file, the only command I am interested now is "stop=0". If "stop" is 1, then exit. [edit: other commands I can implement by myself later]

2. check if "worktodo.txt" exists in the current folder, and if so, check its size. If the size is not zero, skip step 3. [edit: this is because mfaktc does not delete the file when he finishes the assignment, the file is still there, but its size is zero, as no assignment line inside. A nonzero file would mean that the previous assignment was not finished, but accidentally interrupted; edit2: or that the mfaktc is still running on it, see below]

3. if "stop" is zero [edit: obviously is zero, otherwise we won't be here!], fetch a single LL-TF assignment from GPU272 (customizable range/bitdepth would be fine, and I don't mind editing it from time to time, but fixed range and 72/73 bits is also fine, this given by the GPU72 server interface, he can give me one expo he likes, and the bitdepth he likes, I don't mind, something like yoyo's script and factorDB single composite fetching) and write it to worktodo.txt

4. call mfaktc with some parameters, in a specified folder and wait for it to finish (like >>start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1) [edit2: if step3 was skipped, see above, there is deliberately no check if any mfaktc is running, more instances can be launched, and this is responsibility of the user. Let's keep it simple here]

5. report the result to whatever (primenet, gpu72, james' site, etc) (obviously it would need my PrimeNet user name and password, so better not be an exe file or black box. Batch/perl/source preferred).

6. repeat from 1 inclusive.

(the reason of 1 is kind of "graceful shutdown", I edit the ini file and put a "1" instead of 0, he would finish the current assignment and exit. Next time I would need to edit the ini again, to make it 0, but I don't mind).

Is anything like this (or close to it) available?
If so, I plan to test it with 4 or 6 copies/instances for a week.

I can do it by myself, except I don't know how to fetch the TF exponents (I assume I need some support/handshake for it from chalsall's side, and obviously it will need my GPU272 user and password, so not any bot can fetch thousands of assignments and never finish any) and I don't know how to report the results yet (this part can be solved, as there is some spider available, which I saved somewhere, but never used it, I can dig into it if it is a must). However, it would be nice to get a ready-made toy.

[chalsall]

Originally Posted by LaurV

I can do it by myself, except I don't know how to fetch the TF exponents (I assume I need some support/handshake for it from chalsall's side, and obviously it will need my GPU272 user and password, so not any bot can fetch thousands of assignments and never finish any) and I don't know how to report the results yet (this part can be solved, as there is some spider available, which I saved somewhere, but never used it, I can dig into it if it is a must). However, it would be nice to get a ready-made toy.

I have been thinking about this for quite some time -- it's time I start writing code for it. It will be in Perl.

I think I've figured out a graceful way of handing "trust" between the clients and the server. If after implementing and testing it is agreed it's workable, this might be a way of having mfakt* talk to Primenet in a trusted fashion which doesn't require secret code.

[Dubslow]

Originally Posted by chalsall

I think I've figured out a graceful way of handing "trust" between the clients and the server. If after implementing and testing it is agreed it's workable, this might be a way of having mfakt* talk to Primenet in a trusted fashion which doesn't require secret code.

What about Prime95? Is there something special about mfakt*/tf that allows this method to be open source?

[LaurV]

The "special" rely in the things you can do with it. This is not like the bank account. You can't "steal" my work or destroy it, if you know my password. You can see what hardware I have, or some "secrets" which anyhow I boast them everyday on the forum. The only bad thing you can do with my gpu272/p95 password is to report results on my name. Be my guest.

[edit: moreover, P95 does not need your PrimeNet password, there is no place in P95 to put your PrimeNet password, you don't need it to fetch work and report results, and Chris's proxy doesn't know it, and does not need it. You can put in any of your computers the username LaurV and get work on my name, and increase my credit when you report the results. You wont need my password for it ]

[edit2: and to answer your question, the "secret" part in P95 try to forbid you to report fake results (i.e. it deals with assignment keys, shifts, whatever). There is no such issue on mfaktc, a factor is a factor, you can't report a fake factor, and there is nothing to deal with fake "no factor" reports. One can report fake "no factors" and get thousands of TF credit. The point is that those expos will have to be LL-ed anyhow. When I began mfakt-ing I raised this issue on the forum, but people said I am overreacting ]

[chalsall]

Originally Posted by Dubslow

What about Prime95? Is there something special about mfakt*/tf that allows this method to be open source?

No. It's nothing special -- instead of "secret code" generating the secret hash (sh), I plan to have a "shared secret" to generate it. Basically each Computer on GPU72 will have a unique MD5 string which will be fetched by each user and placed in the fetching spider's config file(s). This will then be used to generate a hash which is sent with each message.

The only reason it wouldn't work well with Prime95/mprime is it would mean someone couldn't simply download the program and start running it.

[Batalov]

That would be similar to the BOINC's two keys:

User ID 
(Used in community functions)              11..99 
Account key
(Provides full access to your account)     fcb...666 
Weak account key
(Provides limited access to your account) 147..._...777

[KyleAskine]

If this made it's way to mfakto, that would be amazing.

[chalsall]

Originally Posted by KyleAskine

If this made it's way to mfakto, that would be amazing.

That's the plan. External spiders are simply short-term "duct tape".

"This is only temporary, unless it works..." - Red Green

[chalsall]

Hey all.

OK, I've begun implementing the "fetching spider". I had hoped I would be able to provide Perl code which would work under both Linux and Windows. Unfortunately I've run into a problem I need some help with.

For context, I had planned on having the spider fetch work, build the worktodo.txt file with a single candidate, and then fork() a child which would exec() mfakt*. I want to do this so I can have the fetching spider grab another candidate in parallel, return completed work in parallel, monitor mfakt*, provide the output of mfakt* to the user, be able to kill() mfakt* when needed, etc.

Now I know that Windows doesn't actually have a fork() function, but the good people at ActiveState implemented "forked pseudo-processes". Details at http://perldoc.perl.org/perlfork.html

Unfortunately, while the example work-around code for pipes works fine under Linux, it doesn't seem to work under Strawberry Perl on Windows. I'm wondering if those who use Perl under Windows could give the following code a try, and let me know if it works for them:

#!/usr/bin/perl -w

TestFork(1);
TestFork(2);
exit;


sub TestFork {
   ($Test) = @_;

   if (pipe_from_fork('BAR')) {
      # parent
      while (<BAR>) { print; }
      close BAR;
      print "\n\nTest completed.  Was output seen from the child?\n\n\n";
   } else {
      # child

      if ($Test == 1) {
         for ($Cnt = 0; $Cnt<5; $Cnt++) {
            print "We're here!  Cnt: ${Cnt}\n";
            sleep(1);
         }
         exit(0);
      } else {
         exec("dir");
      }
   }
}


sub pipe_from_fork ($) {
   my $parent = shift;
   pipe $parent, my $child or die;

   my $pid = fork();
   die "fork() failed: $!" unless defined $pid;

   if ($pid) {
      close $child;
   } else {
      close $parent;
      open(STDOUT, ">&=" . fileno($child)) or die;
   }
   $pid;
}

The output should be something like this. The "We're here!" bit is the child process talking to the parent. The "fetch_test.pl" bit is the output of the exec()ed "dir" command, and might be different for you (if it works) as a function of what else is in the directory where you run the program.

We're here!  Cnt: 0
We're here!  Cnt: 1
We're here!  Cnt: 2
We're here!  Cnt: 3
We're here!  Cnt: 4


Test completed.  Was output seen from the child?


fetch_test.pl


Test completed.  Was output seen from the child?

I'm particularly interested in those who run ActiveState's Perl, and/or cygwin's.

If I can't find a solution which works in all Perl environments, then I'll simply have to have different versions -- one for Linux, and one for Windows. But this is supposed to work!!!

(Have I mentioned I hate Windows? How could a modern operating system not provide the simple command "tail"?)

[LaurV]

I don't understand your forking problem... (for-king, that is for me, isn't it?).

I use to start multiple windows processes in batch files using... start command. For example, next batch is "forking" 6 windows with 6 copies of mfaktc, any of them having its own folder, files, options, window title, etc, all run low priority, you can even set process affinity, etc.

start "mfaktc1" /D mf1 /I /LOW mf1\mfaktc1.exe -d 0
start "mfaktc2" /D mf2 /I /LOW mf2\mfaktc2.exe -d 0
start "mfaktc3" /D mf3 /I /LOW mf3\mfaktc3.exe -d 1
start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1
start "mfaktc5" /D mf5 /I /LOW mf5\mfaktc5.exe -d 2
start "mfaktc6" /D mf6 /I /LOW mf6\mfaktc6.exe -d 2

this certainly works in win7-64, what I am using on this comp, and i have variants for xp, vista, etc.

You implement the server side (i.e. accessing a link would pop me up one single line with the expo and bit levels, same as FactorDB pops a composite to factor when you access the right link) and let us deal with mfakting that expo and reporting the results. Well, if you want to do all in one, is even better.

[chalsall]

Originally Posted by LaurV

I don't understand your forking problem... (for-king, that is for me, isn't it?).

I use to start multiple windows processes in batch files using... start command. For example, next batch is "forking" 6 windows with 6 copies of mfaktc, any of them having its own folder, files, options, window title, etc, all run low priority, you can even set process affinity, etc.

start "mfaktc1" /D mf1 /I /LOW mf1\mfaktc1.exe -d 0
start "mfaktc2" /D mf2 /I /LOW mf2\mfaktc2.exe -d 0
start "mfaktc3" /D mf3 /I /LOW mf3\mfaktc3.exe -d 1
start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1
start "mfaktc5" /D mf5 /I /LOW mf5\mfaktc5.exe -d 2
start "mfaktc6" /D mf6 /I /LOW mf6\mfaktc6.exe -d 2

1. Does this work under Linux?

2. Do you have any control of the "started" instances? Can you kill() them? Can you monitor their STDOUT or STDERR output?

Please forgive me for my pedantic nature, but if you're going to write software which will be used by others, it is in your best interest to make sure it will always work correctly.

There's a reason "Have you tried turning it off and on again?" is not funny to some....