mersenneforum.org Automatic fetch of Trial Factoring work for GPU mfakt*
 Register FAQ Search Today's Posts Mark Forums Read

 2012-08-14, 16:53 #1 LaurV Romulan Interpreter     "name field" Jun 2011 Thailand 2×11×467 Posts Automatic fetch of Trial Factoring work for GPU mfakt* Ok, now the fetching of P95 work is solved. I need a toy to fetch TF assignments from GPU272. In fact, what I need the toy/spider/minion/terrorist to do is: 1. read a command (or ini, cfg, etc) file, the only command I am interested now is "stop=0". If "stop" is 1, then exit. [edit: other commands I can implement by myself later] 2. check if "worktodo.txt" exists in the current folder, and if so, check its size. If the size is not zero, skip step 3. [edit: this is because mfaktc does not delete the file when he finishes the assignment, the file is still there, but its size is zero, as no assignment line inside. A nonzero file would mean that the previous assignment was not finished, but accidentally interrupted; edit2: or that the mfaktc is still running on it, see below] 3. if "stop" is zero [edit: obviously is zero, otherwise we won't be here!], fetch a single LL-TF assignment from GPU272 (customizable range/bitdepth would be fine, and I don't mind editing it from time to time, but fixed range and 72/73 bits is also fine, this given by the GPU72 server interface, he can give me one expo he likes, and the bitdepth he likes, I don't mind, something like yoyo's script and factorDB single composite fetching) and write it to worktodo.txt 4. call mfaktc with some parameters, in a specified folder and wait for it to finish (like >>start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1) [edit2: if step3 was skipped, see above, there is deliberately no check if any mfaktc is running, more instances can be launched, and this is responsibility of the user. Let's keep it simple here] 5. report the result to whatever (primenet, gpu72, james' site, etc) (obviously it would need my PrimeNet user name and password, so better not be an exe file or black box. Batch/perl/source preferred). 6. repeat from 1 inclusive. (the reason of 1 is kind of "graceful shutdown", I edit the ini file and put a "1" instead of 0, he would finish the current assignment and exit. Next time I would need to edit the ini again, to make it 0, but I don't mind). Is anything like this (or close to it) available? If so, I plan to test it with 4 or 6 copies/instances for a week. I can do it by myself, except I don't know how to fetch the TF exponents (I assume I need some support/handshake for it from chalsall's side, and obviously it will need my GPU272 user and password, so not any bot can fetch thousands of assignments and never finish any) and I don't know how to report the results yet (this part can be solved, as there is some spider available, which I saved somewhere, but never used it, I can dig into it if it is a must). However, it would be nice to get a ready-made toy. Last fiddled with by LaurV on 2012-08-14 at 17:19 Reason: Made into it's own thread.
2012-08-14, 17:19   #2
chalsall
If I May

"Chris Halsall"
Sep 2002

2B3C16 Posts

Quote:
 Originally Posted by LaurV I can do it by myself, except I don't know how to fetch the TF exponents (I assume I need some support/handshake for it from chalsall's side, and obviously it will need my GPU272 user and password, so not any bot can fetch thousands of assignments and never finish any) and I don't know how to report the results yet (this part can be solved, as there is some spider available, which I saved somewhere, but never used it, I can dig into it if it is a must). However, it would be nice to get a ready-made toy.
I have been thinking about this for quite some time -- it's time I start writing code for it. It will be in Perl.

I think I've figured out a graceful way of handing "trust" between the clients and the server. If after implementing and testing it is agreed it's workable, this might be a way of having mfakt* talk to Primenet in a trusted fashion which doesn't require secret code.

2012-08-14, 17:25   #3
Dubslow

"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3·29·83 Posts

Quote:
 Originally Posted by chalsall I think I've figured out a graceful way of handing "trust" between the clients and the server. If after implementing and testing it is agreed it's workable, this might be a way of having mfakt* talk to Primenet in a trusted fashion which doesn't require secret code.
What about Prime95? Is there something special about mfakt*/tf that allows this method to be open source?

2012-08-14, 17:35   #5
chalsall
If I May

"Chris Halsall"
Sep 2002

22×2,767 Posts

Quote:
 Originally Posted by Dubslow What about Prime95? Is there something special about mfakt*/tf that allows this method to be open source?
No. It's nothing special -- instead of "secret code" generating the secret hash (sh), I plan to have a "shared secret" to generate it. Basically each Computer on GPU72 will have a unique MD5 string which will be fetched by each user and placed in the fetching spider's config file(s). This will then be used to generate a hash which is sent with each message.

The only reason it wouldn't work well with Prime95/mprime is it would mean someone couldn't simply download the program and start running it.

 2012-08-14, 17:41 #6 Batalov     "Serge" Mar 2008 Phi(4,2^7658614+1)/2 274516 Posts That would be similar to the BOINC's two keys: Code: User ID (Used in community functions) 11..99 Account key (Provides full access to your account) fcb...666 Weak account key (Provides limited access to your account) 147..._...777
 2012-08-14, 17:55 #7 KyleAskine     Oct 2011 Maryland 12216 Posts If this made it's way to mfakto, that would be amazing.
2012-08-14, 18:07   #8
chalsall
If I May

"Chris Halsall"
Sep 2002

22·2,767 Posts

Quote:
 Originally Posted by KyleAskine If this made it's way to mfakto, that would be amazing.
That's the plan. External spiders are simply short-term "duct tape".

"This is only temporary, unless it works..." - Red Green

 2012-08-17, 15:26 #9 chalsall If I May     "Chris Halsall" Sep 2002 Barbados 22·2,767 Posts Assistance requested... Hey all. OK, I've begun implementing the "fetching spider". I had hoped I would be able to provide Perl code which would work under both Linux and Windows. Unfortunately I've run into a problem I need some help with. For context, I had planned on having the spider fetch work, build the worktodo.txt file with a single candidate, and then fork() a child which would exec() mfakt*. I want to do this so I can have the fetching spider grab another candidate in parallel, return completed work in parallel, monitor mfakt*, provide the output of mfakt* to the user, be able to kill() mfakt* when needed, etc. Now I know that Windows doesn't actually have a fork() function, but the good people at ActiveState implemented "forked pseudo-processes". Details at http://perldoc.perl.org/perlfork.html Unfortunately, while the example work-around code for pipes works fine under Linux, it doesn't seem to work under Strawberry Perl on Windows. I'm wondering if those who use Perl under Windows could give the following code a try, and let me know if it works for them: Code: #!/usr/bin/perl -w TestFork(1); TestFork(2); exit; sub TestFork { ($Test) = @_; if (pipe_from_fork('BAR')) { # parent while () { print; } close BAR; print "\n\nTest completed. Was output seen from the child?\n\n\n"; } else { # child if ($Test == 1) { for ($Cnt = 0;$Cnt<5; $Cnt++) { print "We're here! Cnt:${Cnt}\n"; sleep(1); } exit(0); } else { exec("dir"); } } } sub pipe_from_fork ($) { my$parent = shift; pipe $parent, my$child or die; my $pid = fork(); die "fork() failed:$!" unless defined $pid; if ($pid) { close $child; } else { close$parent; open(STDOUT, ">&=" . fileno($child)) or die; }$pid; } The output should be something like this. The "We're here!" bit is the child process talking to the parent. The "fetch_test.pl" bit is the output of the exec()ed "dir" command, and might be different for you (if it works) as a function of what else is in the directory where you run the program. Code: We're here! Cnt: 0 We're here! Cnt: 1 We're here! Cnt: 2 We're here! Cnt: 3 We're here! Cnt: 4 Test completed. Was output seen from the child? fetch_test.pl Test completed. Was output seen from the child? I'm particularly interested in those who run ActiveState's Perl, and/or cygwin's. If I can't find a solution which works in all Perl environments, then I'll simply have to have different versions -- one for Linux, and one for Windows. But this is supposed to work!!! (Have I mentioned I hate Windows? How could a modern operating system not provide the simple command "tail"?)
 2012-08-17, 16:05 #10 LaurV Romulan Interpreter     "name field" Jun 2011 Thailand 240428 Posts I don't understand your forking problem... (for-king, that is for me, isn't it?). I use to start multiple windows processes in batch files using... start command. For example, next batch is "forking" 6 windows with 6 copies of mfaktc, any of them having its own folder, files, options, window title, etc, all run low priority, you can even set process affinity, etc. Code: start "mfaktc1" /D mf1 /I /LOW mf1\mfaktc1.exe -d 0 start "mfaktc2" /D mf2 /I /LOW mf2\mfaktc2.exe -d 0 start "mfaktc3" /D mf3 /I /LOW mf3\mfaktc3.exe -d 1 start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1 start "mfaktc5" /D mf5 /I /LOW mf5\mfaktc5.exe -d 2 start "mfaktc6" /D mf6 /I /LOW mf6\mfaktc6.exe -d 2 this certainly works in win7-64, what I am using on this comp, and i have variants for xp, vista, etc. You implement the server side (i.e. accessing a link would pop me up one single line with the expo and bit levels, same as FactorDB pops a composite to factor when you access the right link) and let us deal with mfakting that expo and reporting the results. Well, if you want to do all in one, is even better.
2012-08-17, 16:20   #11
chalsall
If I May

"Chris Halsall"
Sep 2002

22·2,767 Posts

Quote:
 Originally Posted by LaurV I don't understand your forking problem... (for-king, that is for me, isn't it?). I use to start multiple windows processes in batch files using... start command. For example, next batch is "forking" 6 windows with 6 copies of mfaktc, any of them having its own folder, files, options, window title, etc, all run low priority, you can even set process affinity, etc. Code: start "mfaktc1" /D mf1 /I /LOW mf1\mfaktc1.exe -d 0 start "mfaktc2" /D mf2 /I /LOW mf2\mfaktc2.exe -d 0 start "mfaktc3" /D mf3 /I /LOW mf3\mfaktc3.exe -d 1 start "mfaktc4" /D mf4 /I /LOW mf4\mfaktc4.exe -d 1 start "mfaktc5" /D mf5 /I /LOW mf5\mfaktc5.exe -d 2 start "mfaktc6" /D mf6 /I /LOW mf6\mfaktc6.exe -d 2
1. Does this work under Linux?

2. Do you have any control of the "started" instances? Can you kill() them? Can you monitor their STDOUT or STDERR output?

Please forgive me for my pedantic nature, but if you're going to write software which will be used by others, it is in your best interest to make sure it will always work correctly.

There's a reason "Have you tried turning it off and on again?" is not funny to some....

 Similar Threads Thread Thread Starter Forum Replies Last Post DuskFalls GPU Computing 5 2017-12-02 00:34 jfamestad PrimeNet 3 2016-11-06 20:32 lidocorc PrimeNet 4 2008-11-06 18:48 jocelynl 15k Search 0 2003-07-11 14:23 ThomRuley Software 5 2003-05-30 20:34

All times are UTC. The time now is 11:56.

Mon Jan 30 11:56:40 UTC 2023 up 165 days, 9:25, 0 users, load averages: 0.89, 0.78, 0.82