Misfit and worktodo.add with mfaktc
 

Hi. I'm using misfit to fetch work from GPU72. I have a problem where misfit puts work in worktodo.add and then it stays there. And next time misfit wants to add work or it wants to drain MISFITworktodo.txt it complains that worktodo.add have work in it already.

The thing that's bothers me is that it's not constant. Someone it works and if I move the work from .add to .txt and then delete .add it will work at least once. All way from fetch to .txt and then deleting .add.

Have anyone had the same problem or is there something I can check to make it work constant. Do I need to go through .add or is there any option to let misfit add work to .txt directly?

And if anyone can explain more in detail how it is supposed to work I might find what I have done wrong. And what is triggering the work in .add to get moved in to .txt? Is that misfit or mfaktc?
/Thecmaster

[QUOTE=Thecmaster;512780]Hi. I'm using misfit to fetch work from GPU72. I have a problem where misfit puts work in worktodo.add and then it stays there. And next time misfit wants to add work or it wants to drain MISFITworktodo.txt it complains that worktodo.add have work in it already.

The thing that's bothers me is that it's not constant. Someone it works and if I move the work from .add to .txt and then delete .add it will work at least once. All way from fetch to .txt and then deleting .add.

Have anyone had the same problem or is there something I can check to make it work constant. Do I need to go through .add or is there any option to let misfit add work to .txt directly?

And if anyone can explain more in detail how it is supposed to work I might find what I have done wrong. And what is triggering the work in .add to get moved in to .txt? Is that misfit or mfaktc?
/Thecmaster[/QUOTE]Haven't used misfit myself, but:
mfaktc version you're using is v0.21? (0.20 or earlier doesn't check for .add file)
worktodo.add file is created in mfaktc's working directory by misfit?
Next check file permissions, and ownership and privilege of the misfit and mfaktc processes.

[QUOTE=kriesel;512784]Haven't used misfit myself, but:
mfaktc version you're using is v0.21? (0.20 or earlier doesn't check for .add file)
worktodo.add file is created in mfaktc's working directory by misfit?
Next check file permissions, and ownership and privilege of the misfit and mfaktc processes.[/QUOTE]

Yes. I'm using 0.21 and the .add is created in mfaktc directory. I can check privileges and ownership. Haven't done that. I can try to run everything as administrator to.

[QUOTE=Thecmaster;512789]Yes. I'm using 0.21 and the .add is created in mfaktc directory. I can check privileges and ownership. Haven't done that. I can try to run everything as administrator to.[/QUOTE]
what does mfaktc.ini say about check interval?
[CODE]# How often to check for more work to add to worktodo file; 3600 second max

WorkFileAddDelay=3600[/CODE]

[QUOTE=kriesel;512790]what does mfaktc.ini say about check interval?
[CODE]# How often to check for more work to add to worktodo file; 3600 second max

WorkFileAddDelay=3600[/CODE][/QUOTE]


Oooh. I don't know. I have to check that. That can be something to check in to. Thinking if misfit wants to add work more often than mfaktc wants to.

I'm not home before Sunday. I'll report back when I have checked.

[QUOTE=Thecmaster;512791]I'm not home before Sunday. I'll report back when I have checked.[/QUOTE]

kriesel has given you good advise.

What he has described is very similar to what is known as a "race condition", or alternatively a "temporal harmonic". It could well explain why you see this issue sometimes, but not others.

[QUOTE=chalsall;512792]kriesel has given you good advise.

What he has described is very similar to what is known as a "race condition", or alternatively a "temporal harmonic". It could well explain why you see this issue sometimes, but not others.[/QUOTE]

Yeah. I got some solid advice from kriesel. And I'm grateful again for fast responses and all solid advise and help you give me when I can't figure it out my self.

I'll get back when I have checked and also when I got a resault. Someone else could get the same problem in the future and find this thread.

[QUOTE=kriesel;512790]what does mfaktc.ini say about check interval?
[CODE]# How often to check for more work to add to worktodo file; 3600 second max

WorkFileAddDelay=3600[/CODE][/QUOTE]

I don't have WorkFileAddDelay= in my mfaktc.ini Does that meen it are suposed to add rightaway when worktodo.add is detected?

[QUOTE=Thecmaster;513010]I don't have WorkFileAddDelay= in my mfaktc.ini Does that meen it are suposed to add rightaway when worktodo.add is detected?[/QUOTE]

No. If it is not defined, the default is 600 seconds.

With worktodoadddelay=3600, it looks like it checks after each class completion that is more than 3600s past the previous check.
[CODE]Apr 05 11:35 | 2971 64.5% | 644.59 2d13h | 1105.11 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 11:46 | 2976 64.6% | 644.46 2d12h | 1105.33 90677 n.a.%
Apr 05 11:56 | 2979 64.7% | 644.56 2d12h | 1105.15 90677 n.a.%
Apr 05 12:07 | 2980 64.8% | 651.02 2d13h | 1094.18 90677 n.a.%
Date Time | class Pct | time ETA | GHz-d/day Sieve Wait
Apr 05 12:18 | 2991 64.9% | 666.96 2d14h | 1068.04 90677 n.a.%
Apr 05 12:30 | 2992 65.0% | 667.65 2d14h | 1066.93 90677 n.a.%
Apr 05 12:41 | 3000 65.1% | 667.95 2d14h | 1066.46 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 12:52 | 3004 65.2% | 668.18 2d13h | 1066.08 90677 n.a.%
Apr 05 13:03 | 3007 65.3% | 669.06 2d13h | 1064.70 90677 n.a.%
Apr 05 13:14 | 3012 65.4% | 669.63 2d13h | 1063.79 90677 n.a.%
Apr 05 13:25 | 3015 65.5% | 673.00 2d13h | 1058.46 90677 n.a.%
Apr 05 13:44 | 3019 65.6% | 1130.0 4d07h | 630.37 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 14:03 | 3024 65.7% | 1154.5 4d09h | 617.00 90677 n.a.%
Apr 05 14:23 | 3027 65.8% | 1157.2 4d09h | 615.56 90677 n.a.%
Apr 05 14:42 | 3031 65.9% | 1157.5 4d09h | 615.41 90677 n.a.%
Apr 05 15:02 | 3036 66.0% | 1174.5 4d10h | 606.48 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 15:21 | 3039 66.1% | 1194.4 4d11h | 596.38 90677 n.a.%
Apr 05 15:41 | 3040 66.3% | 1194.4 4d11h | 596.38 90677 n.a.%
Apr 05 16:01 | 3052 66.4% | 1195.0 4d11h | 596.09 90677 n.a.%
Apr 05 16:21 | 3055 66.5% | 1194.9 4d10h | 596.15 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 16:34 | 3060 66.6% | 739.37 2d17h | 963.44 90677 n.a.%
Apr 05 17:02 | 3064 66.7% | 1716.0 6d08h | 415.12 90677 n.a.%
Apr 05 17:35 | 3067 66.8% | 1973.4 7d06h | 360.97 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 18:08 | 3075 66.9% | 1973.6 7d06h | 360.93 90677 n.a.%
Date Time | class Pct | time ETA | GHz-d/day Sieve Wait
Apr 05 18:41 | 3084 67.0% | 1976.0 7d05h | 360.49 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 19:14 | 3091 67.1% | 1973.2 7d05h | 361.01 90677 n.a.%
Apr 05 19:47 | 3096 67.2% | 1974.6 7d04h | 360.74 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 20:20 | 3099 67.3% | 1974.5 7d04h | 360.77 90677 n.a.%
Apr 05 20:52 | 3111 67.4% | 1957.4 7d02h | 363.91 90677 n.a.%
checking for "worktodo.add"... not found
Apr 05 21:24 | 3112 67.5% | 1936.6 6d23h | 367.82 90677 n.a.%
Apr 05 21:57 | 3115 67.6% | 1945.9 7d00h | 366.07 90677 n.a.%
checking for "worktodo.add"... not found
[/CODE]

[QUOTE=Thecmaster;513010]I don't have WorkFileAddDelay= in my mfaktc.ini Does that meen it are suposed to add rightaway when worktodo.add is detected?[/QUOTE]

Technically, undefined. You have to drill down on the code to be sure. Is it never, or the default plus some value.

Sorry for being pedantic here, but I'm currently involved in a somewhat serious debate which involves only the word "and" in some code (law).

Seriously. :smile:

Ok. Thanx. Maybe if I add delay it will work batter. Don't know. I'll add and check in later.

This works as it is supposed to. No bugs here. Keep calm and run TF...

Misfit does not use .add file at all, and why should he?

The .add file is used by mfaktc, same way as p95, where the idea was borrowed from. When you need to add work, you must avoid editing worktodo.txt file by yourself. This is because the program itself is editing the worktodo.txt to keep track of the work it does, i.e. lines are deleted when assignments are done. For example, if assignments A, B, C are in the file, and you edit it to add assignment D, then your editor makes a copy of it in RAM, which contain A, B, and C, then you add the line D, but meantime, because you type so slow, :razz: the assignment A is finished and the program will delete the line A from the file (but not from your editor, as it has no idea that somebody is editing the file). Now, the file on disk has B and C, but the file in your editor has A, B, C, D. Therefore, if you save the file now, after adding the line D, the new worktodo file will contain A, B, C, D, and the effect is that you added not only the assignment D, but also A (which will be done two times, wasting resources, as the assignment was done already).

To avoid this, the worktodo.add was introduced with the idea that you can put there the work you need to add, eventually in a separate folder to play with it how long you want, then copy it in the folder where the program is, and [U]next time[/U] when the worktodo.txt is edited by the program, the worktodo.add will be added to it, and deleted. Simple and safe.

Misfit has no idea about it, and it does not touch, nor edit, nor use in any way, this file. But... BUT... a good program like Misfit will check if the .add file exists, and if so, it will not add work to worktodo.txt, and let mfaktc to "integrate" first the work in the .add file, which has higher priority. Otherwise, if misfit will continue add work, with higher priority, it may be that .add is never added. Mind that misfit checks hourly (settable from the config) to add the work, but worktodo.txt is edited by the program every time a TF assignment is finished, which can be every second, or once per day, depending of your exponent and bitlevel - some assignments are very fast, some take ages...

So, if you have a problem with .add file, the problem is with mfaktc, and not misfit. Misfit will wait for mfaktc to "digest" the .add file, before adding new work.

Here on this side of the pool, we check by eyes how long till the mfaktc assignment will finish, and if we consider that we have enough time, we edit it directly. Otherwise if like 5 minutes or less left, we wait first to finish the assignment. So we never need to use the .add file. But we are aware of its existence and behavior. Usually, for the bitlevels we work, the assignments take hours, so no issue. You would really need the .add file when your assignments take seconds, or 2-3 minutes each, and you would not have time to edit the worktodo.txt

On the other hand, if you have misifit, why don't you add the new work directly to the misfit worktodo file? (which is updated hourly or so, and moreover you can "pause automation" to be on the safe side).

[QUOTE=LaurV;513067]This works as it is supposed to. No bugs here. Keep calm and run TF...

Misfit does not use .add file at all, and why should he?
[/quote]
Probably wrong.
1, it is misfits that makes the worktodo.add. more about that later.
2. I didn't say it was a bug. I wanted help with settings to make it work as it should. I figured I had missed something when researching the problem, testing and digging in the forum about it.
[quote]
The .add file is used by mfaktc, same way as p95, where the idea was borrowed from. When you need to add work, you must avoid editing worktodo.txt file by yourself. This is because the program itself is editing the worktodo.txt to keep track of the work it does, i.e. lines are deleted when assignments are done. For example, if assignments A, B, C are in the file, and you edit it to add assignment D, then your editor makes a copy of it in RAM, which contain A, B, and C, then you add the line D, but meantime, because you type so slow, :razz: the assignment A is finished and the program will delete the line A from the file (but not from your editor, as it has no idea that somebody is editing the file). Now, the file on disk has B and C, but the file in your editor has A, B, C, D. Therefore, if you save the file now, after adding the line D, the new worktodo file will contain A, B, C, D, and the effect is that you added not only the assignment D, but also A (which will be done two times, wasting resources, as the assignment was done already).
[/quote]

Yeah. But i did that manually before i started to use misfit. I started to use misfit because i'm away from home days to a week and i don't want mfaktc to ever run out of work. I wanted it automatic for later this summer when i'm away for a few week without being home.

[quote]
To avoid this, the worktodo.add was introduced with the idea that you can put there the work you need to add, eventually in a separate folder to play with it how long you want, then copy it in the folder where the program is, and [U]next time[/U] when the worktodo.txt is edited by the program, the worktodo.add will be added to it, and deleted. Simple and safe.

Misfit has no idea about it, and it does not touch, nor edit, nor use in any way, this file. But... BUT... a good program like Misfit will check if the .add file exists, and if so, it will not add work to worktodo.txt, and let mfaktc to "integrate" first the work in the .add file, which has higher priority. Otherwise, if misfit will continue add work, with higher priority, it may be that .add is never added. Mind that misfit checks hourly (settable from the config) to add the work, but worktodo.txt is edited by the program every time a TF assignment is finished, which can be every second, or once per day, depending of your exponent and bitlevel - some assignments are very fast, some take ages...
[/quote]
This isn't totaly true though. Misfit is the program that makes the .add file. And the default settings for misfit is to use the worktodo.add to assign work to mFaktx. otherwise i wouldn't got this question or problem.

[url]https://gyazo.com/72272dd92d1516d22981058b376f433a[/url]

[quote]
So, if you have a problem with .add file, the problem is with mfaktc, and not misfit. Misfit will wait for mfaktc to "digest" the .add file, before adding new work.
[/quote]

It is probably not wrong with either BUT you are also wrong that it's not Misfit. You got here and wrote stuf that's not right.

[quote]
Here on this side of the pool, we check by eyes how long till the mfaktc assignment will finish, and if we consider that we have enough time, we edit it directly. Otherwise if like 5 minutes or less left, we wait first to finish the assignment. So we never need to use the .add file. But we are aware of its existence and behavior. Usually, for the bitlevels we work, the assignments take hours, so no issue. You would really need the .add file when your assignments take seconds, or 2-3 minutes each, and you would not have time to edit the worktodo.txt

On the other hand, if you have misifit, why don't you add the new work directly to the misfit worktodo file? (which is updated hourly or so, and moreover you can "pause automation" to be on the safe side).[/QUOTE]

I want misfit to make everything when i'm not able to. If i'm away for long time. I don't want to be standing without work.



Then we have your attitude. I don't like it. You wrote stuff here like i was stupid or that I wanted to complain., I just want help. And I have done my research before so I know that my question is legit and probably noone have had the same problem. You are just behaving like an asshole in my opinion. sry but how you wrote it is not nice.

I am using Misfit for years, since the very day Scott sent me the source code, and have never seen worktodo.add on my folders... But well, you may be using a different or newer version, or different settings... In that case, I can't help, sorry. You may drop a PM to Scott.


About my attitude, well, you can cry for you mama or deal with it...

[QUOTE=LaurV;513105]I am using Misfit for years, since the very day Scott sent me the source code, and have never seen worktodo.add on my folders... But well, you may be using a different or newer version, or different settings... In that case, I can't help, sorry. You may drop a PM to Scott.


About my attitude, well, you can cry for you mama or deal with it...[/QUOTE]

As I wrote. I don't think it's a bug. And I didn't complain about any of the programs.

I have dealt with your attitude. No need to cry to mama for me when you are the baby.

[QUOTE=Thecmaster;513111]As I wrote. I don't think it's a bug. And I didn't complain about any of the programs.[/QUOTE]

Don't worry about LaurV. The :razz: emoji is almost his trade-mark...

What you have described is happening suggests this is in fact not a bug, like you said. I like to call it "making friends" with code; understanding how it all works, and interoperates.

LaurV is also, I believe, incorrect about the worktodo.add file.

This was introduced by both Scott and Oliver at the suggestion of several seasoned programmers here, to avoid what is known as a "race condition". The issue is that unlike LL'ing, DC'ing and P-1'ing, one cannot predict when mfaktX will find a factor, and so it is never safe for another program to modify the worktodo.txt file.

99.99999% of the time everything is fine, and I do it manually all the time. But that 0.00001% of the time is a problem. It's sometimes referred to as the "once a year" bug, which programmers hate! :smile:

[QUOTE=Thecmaster;512780]Hi. I'm using misfit to fetch work from GPU72. I have a problem where misfit puts work in worktodo.add and then it stays there. And next time misfit wants to add work or it wants to drain MISFITworktodo.txt it complains that worktodo.add have work in it already.

The thing that's bothers me is that it's not constant. Someone it works and if I move the work from .add to .txt and then delete .add it will work at least once. All way from fetch to .txt and then deleting .add.

Have anyone had the same problem or is there something I can check to make it work constant. Do I need to go through .add or is there any option to let misfit add work to .txt directly?

And if anyone can explain more in detail how it is supposed to work I might find what I have done wrong. And what is triggering the work in .add to get moved in to .txt? Is that misfit or mfaktc?
/Thecmaster[/QUOTE]

Do you have your MISFIT work management parameters set to drain the WorkToDo file appropriately? I have used MISFIT for years and and never used the .add feature.

I have mine set as follows:

Drain MISFITworkToDo when total GHzDs drops below 1500
Total GHzDs to assigh-out during drain 700

And my work fetch parameters are set to

Check MISFITworkToDo every (hours) 1
Fetch when MISFITworkToDo GHzDs drops below 1500

[QUOTE=Chuck;513152] I have used MISFIT for years and and never used the .add feature.

I have mine set as follows:

. . .[/QUOTE]

Ditto for me.

[QUOTE=Chuck;513152]Do you have your MISFIT work management parameters set to drain the WorkToDo file appropriately? I have used MISFIT for years and and never used the .add feature.

I have mine set as follows:

Drain MISFITworkToDo when total GHzDs drops below 1500
Total GHzDs to assigh-out during drain 700

And my work fetch parameters are set to

Check MISFITworkToDo every (hours) 1
Fetch when MISFITworkToDo GHzDs drops below 1500[/QUOTE]

It can have something to do with my settings for how to drain. And how often it will check misfit worktodo file. And how long time the assigned task took. I was at the time doing some 75 bit level. And had werry low settings on misfit when to drain and so. And while worktodo.add is in the folder misfit will not see those GHz days. And probably tried to drain a few times before worktodo.add was added to worktodo. And probably it was me and only me that made a problem for my self
Without realising it. Probably I was the one that got to the PC while misfit was waiting for mfaktc to add the .add to .txt I though it was something wrong and put my fingers in the game and giggled it all up.


I have adjusted the numbers and I have done some 74 bit level and it seems to be working as it should. Misfits assigns work, the .add gets consumed and misfit fetch work from gpu72 in to MISFITworktodo file.

I will let it run and see if it works add it should and it is like I think me that made the mishap.

I might put in your numbers in misfit and run that.

Thanx for all help. It is with your help and advise I realised it was probably all me that made the bug. And this have been a week with some learning. Not just what's in this thread.

/Arvid

I didn't give my fetch parameter in the previous post.

GHzDs to fetch 1000

And I Let GPU72 decide

I don't know why you would use the .add feature unless you had some specific tests you wanted to manually put in the MISFITWorkToDo file.

FYI

It seems to work fine now. I haven't got the same problem again after adjusting numbers when to assign work, drain and fetch. I didn't put in the delay line in mFaktc.ini. worktodo.add works add it should to.

I did some adjustments yesterday just. I went from fetching 15 assignments to fetch 1500Ghz days. And drain 700 when below 300.

That way it will fetch and assign reasonable amount what ever bitlevel I'm doing.

Right now I do 74-75 in 92M range.

Thanx for all help. Your information (even if it wasn't exalt the solution) it helped me find a setting that worked and didn't give me the conflict I got.