mfaktc: a CUDA program for Mersenne prefactoring

[bcp19]

Originally Posted by nucleon

This is why that specific ram drive is pretty cool.

It saves an image every x-seconds you specify. It loads the image on bootup. It also as a backup in case power failure during an image write. It's a lot safer now than the earlier versions.

Also as a secondary precaution, I rsync all mfaktc instances to my nas every hour.

Worst case for me I lose say 2 hours. I've been in this unit for about 2months - no power issues for me.

But if you're having a power issue <1/fortnight, I think you have greater issues :)

Another benefit - you can migrate the image to another PC if required. Complete file structure ready to go.

-- Craig

I don't have power issues, I just tend to think about the worst case scenario. I've probably had the power go out twice in the last year, so it's not a big deal.

I didn't look at all the tabs, that is pretty neat, thanks.

[kladner]

Originally Posted by Dubslow

If kladner's around somewhere, he knows a lot about the cmd commands to open mfaktc/CUDALucas (read: he spent a lot of time experimenting).

Hello PsycO: I just noticed this discussion. I use CMD and the Start command to run mfaktc from batch files. I think that this can also be done with a shortcut, but I have not experimented very much with that. My understanding from others here is that this only works completely in Windows 7 or Vista 64 bit versions.

There is more discussion here:
http://www.mersenneforum.org/showpos...0&postcount=77

I currently run three instances of mfaktc, so there are three slightly different command lines to set different core affinities.

The command lines are:

cmd.exe /k "start /b /low /affinity 0x20 mfaktc-win-64.exe"

cmd.exe /k "start /b /low /affinity 0x10 mfaktc-win-64.exe"

cmd.exe /k "start /b /low /affinity 0x08 mfaktc-win-64.exe"

These run mfaktc on cores 4,5, and 6. As you can see, the core affinities are specified by the hex values. Core numbering starts with 0.

I have intended to set up a batch which would start all of these together, but my initial attempts did not work correctly. At the moment I run three different batch files.

My machine remains fairly usable in most situations with the following settings in mfaktc.ini:

NumStreams=3
CPUStreams=3
GridSize=3
AllowSleep=1

AllowSleep=1 frees a small amount of CPU time on the three cores which are feeding mfaktc. Reducing NumStreams to 2 makes the system more responsive without too much impact on mfaktc performance. Reducing GridSize to 2, or even 1, improves responsiveness a lot, but at considerable cost to mfaktc.

For gaming you are probably better off just stopping mfaktc. Unlike P-1 factoring in P95, there does not seem to be a large penalty in restarting it.

[kar_bon]

Originally Posted by kladner

The command lines are:

cmd.exe /k "start /b /low /affinity 0x20 mfaktc-win-64.exe"

cmd.exe /k "start /b /low /affinity 0x10 mfaktc-win-64.exe"

cmd.exe /k "start /b /low /affinity 0x08 mfaktc-win-64.exe"

These run mfaktc on cores 4,5, and 6. As you can see, the core affinities are specified by the hex values. Core numbering starts with 0.

I have intended to set up a batch which would start all of these together, but my initial attempts did not work correctly. At the moment I run three different batch files.

On my WinXP machine I run a batch for starting 8 instances of LLRnet in different directories like this:

@echo off
start "LLRnet Client1" /DLLRnet1 do.bat
start "LLRnet Client2" /DLLRnet2 do.bat
start "LLRnet Client3" /DLLRnet3 do.bat
rem (...) more clients analogous
exit

So the batch starts all LLRnet clients in an own DOS-Box and will close itself.

The do.bat's in those directories are starting every LLRnet client and could be other programs, too.

[kladner]

Originally Posted by kar_bon

On my WinXP machine I run a batch for starting 8 instances of LLRnet in different directories like this:

@echo off
start "LLRnet Client1" /DLLRnet1 do.bat
start "LLRnet Client2" /DLLRnet2 do.bat
start "LLRnet Client3" /DLLRnet3 do.bat
rem (...) more clients analogous
exit

So the batch starts all LLRnet clients in an own DOS-Box and will close itself.

Thanks, kar_bon! When I attempted this I was using the "Call" command to start the individual batch files. This caused the master batch to stop and wait after the first batch. I'll try using "Start".
...............

And indeed, using "start" did the trick!
Thanks, again!

[TheJudger]

Hello!

Originally Posted by PsycO

But i have done a hardware hack to be able to plug it in a 2xPCI-e, so maybe the transfert rate are for something here.

PCIe 2x? Assuming worst case (2.5GT/s) that is 500MB/sec, with 80% bandwidth efficiency this is enough for 100M FCs per second.

Originally Posted by PsycO

If you are interested in a GUI for mfaktc/o, let me know, i will be please to make a universal launcher, for win and linux, that will handle the configuration and launching of the exe. It will detect the number of core and deal with the cmdline options. So let me know if someone is interested, i will be please to participate to the project!!

Well, why not giving a try? To be honest I don't think that I'll use that GUI but other users might have a different opinion.

Originally Posted by PsycO

Another question is, does mfaktc do a lot of IO, on a temp file or to save backup??

If enabled mfaktc writes a checkpoint file every time a class is finished. The file is ~50 bytes big. Perhaps I should add a minimum delay between two checkpoint writes (e.g. 30 seconds or so).

Originally Posted by PsycO

A last thing, for the pause button of the keyboard, do you know pressing the button issue what signal to windows(the shell)?? I wanna know that, this will surely have utility in one of my future project!!!

Does the pause key work for you? Anyway I'll see if there is a simple and save way to do so.

Oliver

[James Heinrich]

Originally Posted by TheJudger

If enabled mfaktc writes a checkpoint file every time a class is finished. The file is ~50 bytes big. Perhaps I should add a minimum delay between two checkpoint writes (e.g. 30 seconds or so).

:surprised
No perhaps, please(!) add in some code to limit the frequency of writing checkpoint files! I didn't realize such murderous I/O was involved. Not in number of bytes written, but in number of writes. I'd suggest making it .ini configurable with a default interval of 60 seconds, maybe even 300 seconds. Nobody's going to care if they lost one minute of work (it'll take that long to notice and restart mfaktc anyway), it's where the assignment has been running many hours that you care about losing work.

[PsycO]

No perhaps, please(!) add in some code to limit the frequency of writing checkpoint files! I didn't realize such murderous I/O was involved. Not in number of bytes written, but in number of writes. I'd suggest making it .ini configurable with a default interval of 60 seconds, maybe even 300 seconds.

I'm totally with James Heinrich for this point, a delay of 300 sec or like then write at the beginning of the next class will be enough. Just make a thread that will do the job, this will eliminate code in the loop!! Less code in loop == faster code!!! Just eliminating these IO's will speed up the app a lots! IO's are very consuming!!

PCIe 2x? Assuming worst case (2.5GT/s) that is 500MB/sec, with 80% bandwidth efficiency this is enough for 100M FCs per second.

Perfect, this elimitate one of my potential bottleneck!

Well, why not giving a try? To be honest I don't think that I'll use that GUI but other users might have a different opinion.

As i see from you and the others, nobody really want it, so i'll not waste my time! If you have somethings else i can do in Python/Tk, let me know. I have a base in C that is enough to review correct already done code, but not enough to make complex things.
If i can know the info's about Primenet protocol's, i can write a module that will deal with it! Maybe with this feature, the GUI will worth the time to make it!

Does the pause key work for you? Anyway I'll see if there is a simple and save way to do so.

Don't waste your time, the pause button pause the app in the second, freeing available resource to play a game or whatever else, and after, press start and the app restart where it left without any problem!! No need to implement a function to do it! If a control is needed in the future, just find the control char that represent the pause on the keyboard and write it to the stdin of the console!

[Dubslow]

Originally Posted by PsycO

If i can know the info's about Primenet protocol's, i can write a module that will deal with it! Maybe with this feature, the GUI will worth the time to make it!

Is this what you were referring to?

[RichD]

Originally Posted by TheJudger

If enabled mfaktc writes a checkpoint file every time a class is finished. The file is ~50 bytes ...

That's good to know, which begs the next question. Is a checkpoint file written when a ctrl-c (break) is invoked?

(I can afford to take the 30-60 minute lost processing [on rare occasions] to gain a few percentage points in processing. My power situation is fairly stable.)

[LaurV]

Originally Posted by kladner

cmd.exe /k "start /b /low /affinity 0x20 mfaktc-win-64.exe"
etc <snip>

you don't really need to make it so complicate, unless you want to specify express which CPU core is used for which copy, and which copy to run on which GPU, etc, the -d switch works perfectly in mfaktc. I use something like:

mfaktc1 -d 0
mfaktc2 -d 0
mfaktc3 -d 0
mfaktc4 -d 1
mfaktc5 -d 1
mfaktc6 -d 1

to launch 6 copies of mfaktc on two GPU's, win7 is enough intelligent to split the CPU burden. The 6 mfaktcX.exe are all identical copies of mfaktc-win-64.exe, placed in different subfolders, each with its own ini and worktodo file. The advantage is that I can customize better the inputs, adding work only to the thread that need (they are not always running at the same speed, depending on the assignments and the CPU/GPU burden for the daily real-life job). The input files can be customized in the ini file, so they can all be in the parent folder, and you don't need to walk to each subfolder to add work. Just look at their size in explorer/totalcmd and you know which one is ready to go dry. Renaming the exe files has also the advantage that you know in every moment what copy of mfaktc does what, because the name appears in the title of the cmdprompt window. Of course this can also be realized on your version if you add the "title" parameter for "start" command (there is an additional parameter where you can specify window's title). The disadvantage of my method is that the results files will be in the subfolders. This can not be (yet) customized in the ini files. But for that I made a batch file to collect all the result.txt from subfolders, so I don't need to walk on each subfolder and look for them.

[PsycO]

Originally Posted by Dubslow

Is this what you were referring to?

Yes this is exactly what i was referring to!!!
But... Woaaaaaaa, tons of work here!!! The server is really complete!!!
Feasable, but i need to know if integration to Prime95 is planned, on a medium time scale, or in the next year??? If not, the project is interesting....
TheJudger: Do you have talked to Woltman on a possible integration??? And do you are the CUDALucas develloper(For permission)?? As they are develloped for the same goal, the ability to crunch for mersenne on GPU, if i make GUI with primenet integration, i will surely integrate them all(mfaktc, mfakto, CUDALucas)!!!