proof errors

[preda]

Originally Posted by aheeffer

I now understand what happened. The same thing as with another exponent I reported about in the gpuowl thread.

There was a CR/LF missing at the end of the 'worktodo.txt'-file contained in the 'pool' folder. It happened to me before and then gpuowl just stops. In this case, it looked at the 'worktodo.txt' file in the local folder and started the same exponent again. Having found the proof folder, it complained about the missing checkpoints.

In the other case, it found an old local 'worktodo.txt' with an expired exponent. I lost a few days work.

Sorry about this.

OK I understand where this is coming from. I'll need to revisit the worktodo.txt code and harden it against lines without newlines.

[preda]

Originally Posted by aheeffer

I now understand what happened. The same thing as with another exponent I reported about in the gpuowl thread.

There was a CR/LF missing at the end of the 'worktodo.txt'-file contained in the 'pool' folder. It happened to me before and then gpuowl just stops. In this case, it looked at the 'worktodo.txt' file in the local folder and started the same exponent again. Having found the proof folder, it complained about the missing checkpoints.

In the other case, it found an old local 'worktodo.txt' with an expired exponent. I lost a few days work.

Sorry about this.

I tried to reproduce this (worktodo.txt with missing newlines), but I can't. Can you reproduce it? if you can, please provide the steps clearly for your repro.

In my case, when I run without newlines, I get this expected behavior:

2020-10-24 10:35:37 XFXsec /home/preda/pool/worktodo.txt : line "PRP=XXX,1,2,100532479,-1,76,2" does not end with a newline
2020-10-24 10:35:37 XFXsec Exiting because "lines must end with newline"

[retina]

Originally Posted by preda

lines must end with newline

Why so OCD?

Just place a LF (or CRLF) on the end of every text file you load into memory. Then you don't have to treat the user to an unnecessary error.

[preda]

Originally Posted by retina

Why so OCD?

Just place a LF (or CRLF) on the end of every text file you load into memory. Then you don't have to treat the user to an unnecessary error.

There is a problem when appending new lines at the end of worktodo.txt, which is normally done simply (like ">>" on Unix, "append to end of file") without having to worry about anything.

OTOH if it's not guaranteed that the file ends with \n, you have two options:
1. always add a '\n' just to be sure -- then the user is surprised by the many empty lines.
2. read and analyze the file, and take different actions depending on how it's ended.

[retina]

Originally Posted by preda

1. always add a '\n' just to be sure -- then the user is surprised by the many empty lines.

I meant to only add it in memory, not on disk.

But anyhow, I'm sure the user is more surprised to get an error when the text file looks perfectly fine to them.

[preda]

Originally Posted by retina

I meant to only add it in memory, not on disk.

But anyhow, I'm sure the user is more surprised to get an error when the text file looks perfectly fine to them.

If I only add in memory, it is not possible to append new lines to the file by simple concatenation.

If the file ends with "AAA" without newline, and I append "BBB\n", now the file contains "AAABBB\n" as the last line. That's not what anybody wants.

[retina]

Originally Posted by preda

If I only add in memory, it is not possible to append new lines to the file by simple concatenation.

If the file ends with "AAA" without newline, and I append "BBB\n", now the file contains "AAABBB\n" as the last line. That's not what anybody wants.

If you are rewriting the file then you can do whatever you want. It is only upon reading, to make the code simple, to add a newline so there doesn't need to be any special handling for just the last line.

If you just want to make the users mad then you can make the code super fussy and generate error conditions like you do now. But to me it seems silly to tell the user to edit the file and put in a LF, when the code could do that itself without any fuss and then keep working as expected.

[aheeffer]

Originally Posted by preda

I tried to reproduce this (worktodo.txt with missing newlines), but I can't. Can you reproduce it? if you can, please provide the steps clearly for your repro.

In my case, when I run without newlines, I get this expected behavior:

The problem is very easy to replicate. Just cut one or more lines from the manual assignments server page and past them into the worktodo.txt file. When reading the file, gpuowl-win.exe will stop unless you hit enter at the last line before saving. There may be differences between Windows and Linux in handling CR/LF.

[LaurV]

Originally Posted by aheeffer

There may be differences between Windows and Linux in handling CR/LF.

Yep. A single CR or LF may work in Linux, but not in Windows, which expects both. Interesting enough, in my computer (Win 7) I was mesmerized to see that two worktodo files which had the same number of P-1 assignments (taken in bunch of hundreds or so, from GPU72, and pasted to worktodo directly, in each folder where the Owl was running. ctrl+c, ctrl+v, using notepad++) had different lengths (by few hundred bytes). I worried that I made a mistake with copy/paste and I counted the lines (i.e. open files with notepad++ which shows line numbers), and they had the same number of lines, and each line had the same number of chars. Totally puzzled!

But when opened in binary mode, one shows 0x0D, 0x0A at the end of each line, the other only had 0x0A.

To my surprise, after a while, they "reverted" (the longer file became shorter), albeit the cards were crunching with the same speed, deleting lines from the files with the same frequency, and I didn't open the files for edit. So, the Owl does indeed make some tricks, when rewriting the files.

But I never got an error! (related to this subject).

(and I always DO insert the last <enter> by hand when I paste assignments, this habit precedes the Owl, due Mfaktc and cudaLucas also needed <enter> or an empty line before the EOF, sometime in their past)

[preda]

Originally Posted by aheeffer

The problem is very easy to replicate. Just cut one or more lines from the manual assignments server page and past them into the worktodo.txt file. When reading the file, gpuowl-win.exe will stop unless you hit enter at the last line before saving. There may be differences between Windows and Linux in handling CR/LF.

It's not clear what you mean by "will stop".

If it throws an exception telling you to add a newline, that's fine.

OTOH what you seem to have reported earlier was an exponent that was ran twice. That's not consistent with "will stop", and that's what I can't reproduce.

In other words: do you see a problem with the current behavior? if so, what is the problem you see, exactly?

[preda]

Originally Posted by retina

If you are rewriting the file then you can do whatever you want. It is only upon reading, to make the code simple, to add a newline so there doesn't need to be any special handling for just the last line.

If you just want to make the users mad then you can make the code super fussy and generate error conditions like you do now. But to me it seems silly to tell the user to edit the file and put in a LF, when the code could do that itself without any fuss and then keep working as expected.

Well my point was that I'm not rewriting the file. As you can imagine, re-writing the file has its own drawbacks, involving files being renamed and deleted and whatnot, so if all I want to do is to append a line to the end of the file, than I don't rewrite the file but I simply append the line.

BTW, it is normally accepted that the defition of a line in a file is text that ends with a line separator. This can be seen in the output of "wc -l" which counts the lines in a file, which does not count as a line text that's not line-ended.

Anyway putting forward such seemingly obvious solutions as "rewriting the file" and "always add a newline on read" and assuming that I don't implement them due to my simple pleasure in tormenting the user.. yep that's it.