[QUOTE=chalsall;296761]From the perspective of a programmer, please explain why.[/quote]I prefer to know that exponents are added exactly in the order I want them ordered. With worktodo.add the order of adding is at the mercy of the programmer. (I realize I'm not the "typical" use case, but that doesn't change the fact that the "typical" solution still bugs me.)
[QUOTE=chalsall;296761]
Agreed. But without IPC (for example, a file lock or some other semaphore) you still have a race condition if any program other than CUDALucas touches that file.
[/quote]How? When CUDALucas finishes the test, it opens the file again, and reads the first "good" assignment out, closes the file again, and works on that. Thus you can modify the file however you want in between, and CUDALucas will always test the first line in the file. (You could get a race condition if you modify the file while it's between tests, but that lasts for less than a second, but that is easily avoidable if the user just uses a little common sense, or a lock/semaphore could solve that minor issue programming-side as well, and that lock/semaphore would only exist for less than a second, as opposed to existing the whole time that CUDALucas is running.)

Note that I'm 99.999% sure that v2.00a, with the hacked mfaktc functions, [i]does[/i] close the file during tests; it uses a get_next_assignment() function that closes the file after getting a good assignment. (It would be trivial to check this by downloading the mfatkc source, and I'll check it myself later today.)

[QUOTE=Dubslow;296762]How? When CUDALucas finishes the test, it opens the file again, and reads the first "good" assignment out, closes the file again, and works on that. Thus you can modify the file however you want in between, and CUDALucas will always test the first line in the file. (You could get a race condition if you modify the file while it's between tests, but that lasts for less than a second...)[/QUOTE]

That's the race condition I'm talking about.

Remember I've talked before about the "once a month" or "once a year" bug. This is an example of how such bugs manifest. The programmer thinks "oh, that won't happen very often"...

And, as an aside, the "once a month" bug issue was really driven home for me a few years ago when I was deploying a wireless network here in Barbados. I was using Linux boxen with ATM cards as base stations. The ATM kernel driver had a bug which would manifest about once a month, locking up the entire machine.

This became a real problem when we had thirty base stations deployed.... :cry:

[QUOTE=chalsall;296764]That's the race condition I'm talking about.

Remember I've talked before about the "once a month" or "once a year" bug. This is an example of how such bugs manifest. The programmer thinks "oh, that won't happen very often"...
[/QUOTE]It's an easily solvable race condition, either with common sense on the user's part, or a lock from the program/mer. The advantage here over worktodo.add is that the lock would be [i]very[/i] short, only the time it takes to read in an assignment, as opposed to being locked for the entire duration of running the program (i.e., indefinitely) with the .add method.

[QUOTE=Dubslow;296765]It's an easily solvable race condition, either with common sense on the user's part...[/QUOTE]

Assuming that a human is in the loop...

[QUOTE=Dubslow;296765]... or a lock from the program/mer. The advantage here over worktodo.add is that the lock would be [i]very[/i] short, only the time it takes to read in an assignment, as opposed to being locked for the entire duration of running the program (i.e., indefinitely) with the .add method.[/QUOTE]

You might be forgetting that file locking is a cooperative agreement (a type of semaphore) between the programs which use the file(s) in question. And file locking doesn't work on some file systems (for example, many network file systems).

Additionally, having CUDALucas close the file while it works does not really enter the equation from the perspective of a fully defined automated run-time environment.

To achieve your particular requirements (even if it closed the file when not needed), you would need to slave CUDALucas under another program, and "kill" it inbetween modifications to the worktodo.txt file, then restart it. (Or, have a human in the loop. But humans are so damn unreliable.... :wink:)

[QUOTE=chalsall;296767]
You might be forgetting that file locking is a cooperative agreement (a type of semaphore) between the programs which use the file(s) in question. [/quote]This problem is not specific to my solution, but also is a problem for worktodo.add, and again, I think that my way is safer because the lock time is (much) shorter.[QUOTE=chalsall;296767] And file locking doesn't work on some file systems (for example, many network file systems).[/quote]Again, this also applies to worktodo.add.[/quote]
[QUOTE=chalsall;296767]
Additionally, having CUDALucas close the file while it works does not really enter the equation from the perspective of a fully defined automated run-time environment.[/quote]I'm not sure what you mean by "fully defined automated run-time environment" (this is just my own ignorance of this term).
[QUOTE=chalsall;296767]
To achieve your particular requirements (even if it closed the file when not needed), you would need to slave CUDALucas under another program, and "kill" it inbetween modifications to the worktodo.txt file, then restart it. (Or, have a human in the loop. But humans are so damn unreliable.... :wink:)[/QUOTE]I don't understand this. My requirements are that I can modify worktodo.txt as I like while CUDALucas is running an assignment, and that when it needs work, it reads the first/top assignment in worktodo.txt at that time. I don't see why I would need to slave it to another program--closing the file is only intended to make sure it gets the most recent version without clobbering any changes I made while it was working on the previous test.

[QUOTE=Dubslow;296772]This problem is not specific to my solution, but also is a problem for worktodo.add, and again, I think that my way is safer because the lock time is (much) shorter. Again, this also applies to worktodo.add.[/QUOTE]

Incorrect.

If you have a program which reads and writes a file ("worktodo.txt" in this case), plus reads another file and then deletes it (if it exists; "worktodo.add"), then you can create a system where another program can write to the second file in such a way that everything is sane.

One simple solution is the writer to the second file checks to see if worktodo.add exists. If does, then it simply exits. Or, it moves the worktodo.add file to a temporary location, and waits for some period of time. If it still exists, then it merges what it wants to add, and moves the file back.

But short of some form of cooperation (like a worktodo.add facility, passing data through an SQL table, etc), without stopping the first program it can never be entirely safe to modify worktodo.txt.

And all of this assumes that there will only be one writer to worktodo.add.

[QUOTE=Dubslow;296772]I'm not sure what you mean by "fully defined automated run-time environment" (this is just my own ignorance of this term).[/QUOTE]

Probably not; it's my own term. It's meant to be descriptive, not a formal or official term.

[QUOTE=Dubslow;296772]I don't understand this. My requirements are that I can modify worktodo.txt as I like while CUDALucas is running an assignment, and that when it needs work, it reads the first/top assignment in worktodo.txt at that time. I don't see why I would need to slave it to another program--closing the file is only intended to make sure it gets the most recent version without clobbering any changes I made while it was working on the previous test.[/QUOTE]

We're looking at this from different perspectives.

You want to be able to be able to modify worktodo.add in the manner you desire, knowing that you are able to observe the state of CUDALucas, and know that you have enough time to do so based on how much longer CUDALucas is going to take.

I'm looking at it from the perspective of how does one create a program which modifies another program's file(s) without the knowledge of when the second program is going to modify the file(s).

This is a difficult problem which occurs a lot. Think Mail Transport Agents (MTAs), for example. Or, closer to "home", how about "mfakt*" where the potential writer (human or 'bot) has no way of knowing when the file will be modified because a factor could be found at any time.

[QUOTE=chalsall;296774]Incorrect.

If you have a program which reads and writes a file ("worktodo.txt" in this case), plus reads another file and then deletes it (if it exists; "worktodo.add"), then you can create a system where another program can write to the second file in such a way that everything is sane.

One simple solution is the writer to the second file checks to see if worktodo.add exists. If does, then it simply exits. Or, it moves the worktodo.add file to a temporary location, and waits for some period of time. If it still exists, then it merges what it wants to add, and moves the file back.

But short of some form of cooperation (like a worktodo.add facility, passing data through an SQL table, etc), without stopping the first program it can never be entirely safe to modify worktodo.txt.

And all of this assumes that there will only be one writer to worktodo.add.
[/quote]I mostly don't understand this. If the first program is not using the file in any way, how can it be unsafe to modify worktodo.txt?
[QUOTE=chalsall;296774]
We're looking at this from different perspectives.

You want to be able to be able to modify worktodo[strike].add[/strike] [i].txt[/i] in the manner you desire, knowing that you are able to observe the state of CUDALucas, and know that you have enough time to do so based on how much longer CUDALucas is going to take.[/quote]Right, given that what I corrected was, in fact, a typo...
[QUOTE=chalsall;296774]
I'm looking at it from the perspective of how does one create a program which modifies another program's file(s) without the knowledge of when the second program is going to modify the file(s).

This is a difficult problem which occurs a lot. Think Mail Transport Agents (MTAs), for example. Or, closer to "home", how about "mfakt*" where the potential writer (human or 'bot) has no way of knowing when the file will be modified because a factor could be found at any time.[/QUOTE]Okay, fair.
But, in CUDALucas' case, we [i]do[/i] know exactly when CUDALucas will modify a file (and if it didn't delete test history, it wouldn't do any writing at all, just reading). That's why I'm saying I don't like worktodo.add in this case, because CUDALucas doesn't fall into this class of program that requires worktodo.add. (That's why I've been continuing this debate -- because this is the CUDALucas thread, so I assumed we were excepting this class of program.) (This sort of reasoning also applies to Prime95 with all GIMPS worktypes except TF, which is why it frustrates me that I have to kill it to modify worktodo.txt.)

[Somewhat OT]
Additionally, with mfaktc, the added benefit (for me) of being able to modify worktodo.txt out of order (i.e. not with .add) outweighs the risk that mfaktc will write to worktodo.txt, given that factors are printed on screen (and the comment about deleting history also applies here). That's why I didn't like the talk about having mfakt* apply a lock indefinitely; at the very least I'd prefer that to be a disable-able option.[/OT]

[QUOTE=Dubslow;296775]I mostly don't understand this. If the first program is not using the file in any way, how can it be unsafe to modify worktodo.txt?[/QUOTE]

Because you don't know when it might open it for reading/writing again.

Think about it from the perspective of another program which has no knowledge of the state of the first program. Further, you're assuming that even if the first program doesn't have the file open that it is prepared to deal with the contents changing.

[QUOTE=Dubslow;296775]Right, given that what I corrected was, in fact, a typo...[/QUOTE]

Yes, I meant to write "worktodo.txt".

[QUOTE=Dubslow;296775]But, in CUDALucas' case, we [i]do[/i] know exactly when CUDALucas will modify a file (and if it didn't delete test history, it wouldn't do any writing at all, just reading).[/QUOTE]

Only by observing it's state. Much harder to do automatically.

[QUOTE=Dubslow;296775]That's why I'm saying I don't like worktodo.add in this case, because CUDALucas doesn't fall into this class of program that requires worktodo.add. (That's why I've been continuing this debate -- because this is the CUDALucas thread, so I assumed we were excepting this class of program.) (This sort of reasoning also applies to Prime95 with all GIMPS worktypes except TF, which is why it frustrates me that I have to kill it to modify worktodo.txt.)[/QUOTE]

But I would argue you have very particular desires, which not everyone shares. Thus, I think that worktodo.add functionality would be useful to many people. It doesn't have to be used, but it would be a good thing to have for many people.

[QUOTE=Dubslow;296775]Additionally, with mfaktc, the added benefit (for me) of being able to modify worktodo.txt out of order (i.e. not with .add) outweighs the risk that mfaktc will write to worktodo.txt, given that factors are printed on screen (and the comment about deleting history also applies here). That's why I didn't like the talk about having mfakt* apply a lock indefinitely; at the very least I'd prefer that to be a disable-able option.[/QUOTE]

But we come back to the "once a month bug". What you want to do will result in failure some times. Rarely, I agree, but sometimes. Sorry for berating this topic, but I've been taught (the hard way) to try to avoid such situations at all times.

And, lastly, I think you misunderstand what was being discussed for the file locking for mfakt*. It was intended to exist only when the file was being modified, not all the time. However, as was pointed out, some use network file systems for their clusters, so file locking would not be a complete solution.

[QUOTE=chalsall;296777]Because you don't know when it might open it for reading/writing again.

Think about it from the perspective of another program which has no knowledge of the state of the first program. Further, you're assuming that even if the first program doesn't have the file open that it is prepared to deal with the contents changing.

Only by observing it's state. Much harder to do automatically.
[/quote]Ah, okay. Another case of crossed based assumptions: I've been thinking that only I would be modifying worktodo.txt.
I can see that if one were to design a spider to get work for mfat[co]/CuLu, then yes I see you would need worktodo.add anyways (and in that case you wouldn't be adding out of order).
[QUOTE=chalsall;296777]
But I would argue you have very particular desires, which not everyone shares. Thus, I think that worktodo.add functionality would be useful to many people. It doesn't have to be used, but it would be a good thing to have for many people.[/quote]Yes, all I ask is to be sure that "It doesn't have to be used" is true, hence my comment about disable-ability.
[QUOTE=chalsall;296777]
And, lastly, I think you misunderstand what was being discussed for the file locking for mfakt*. It was intended to exist only when the file was being modified, not all the time. However, as was pointed out, some use network file systems for their clusters, so file locking would not be a complete solution.[/QUOTE]Ah, okay. Whoops.

Debate resolved with agreement! (I think!) :fusion:
We need a high five smiley.

[QUOTE=Dubslow;296782]Debate resolved with agreement! (I think!) :fusion:[/QUOTE]

Agreed.

[QUOTE=Dubslow;296782]We need a high five smiley.[/QUOTE]

How about :chalsall:? :wink:

[QUOTE=LaurV;296588]Quote:
Originally Posted by [B]Dubslow[/B] [URL="http://www.mersenneforum.org/showthread.php?p=296584#post296584"][IMG]http://www.mersenneforum.org/images/buttons/viewpost.gif[/IMG][/URL]
[I]Anyone want to run it through Prime95? (Edit: Expo is 26409557)

[/I]
I will queue it second on my list (after current DC expo finishes in 6-7 hours) on one core. ETA ~3 days. Should I automatically report it? Or check with you first? (in this case please remind me in few days, otherwise it may remain unreported).[/QUOTE]
I finished and reported, the P95 residue is indeed different, something with 060CAD...