gpuowl has generated 3 proof files but no results.JSON.txt

[Reed_Young]

I've been using PrimeNet for 10+ years without much trouble, but I'm new to the gpuowl software as of December and as far as I can tell, the required results.JSON.txt files are not being created and won't, unless/until I do something, I'm not entirely sure what.

I do have ###,###,###-8.proof files in the proof folder, but no results.JSON.txt file in the directory above that, which is just called gpuowl. The gpuowl directory does contain a results.txt file with information about 1 of the completed exponents, and the gpuowl2 directory contains a results.txt file with information about the other two.

I have copied the ###,###,###-8.proof files into my main Mersenne Primes folder, which contains the mprime executable. My (admittedly, partial) understanding of one thread was that this might be sufficient, as it will eventually cause the server to be notified that the proof files exist and then the results.JSON.txt file (which does exist in that folder, just not in the gpuowl folders) will be updated to include the 3 results that were crunched using gpuowl. Is this correct so far? Or is it also necessary to copy and paste the contents of the results.txt files in the gpuowl directories into the results.txt file in the mprime directory?

I did not use the -pool option when running gpuowl on any of the 3 PRP tests that were completed using gpuowl, but since reading about that option while skimming the forums for a solution to this problem, I am using the -pool option now, to point gpuowl to the folder that contains mprime. Will this be sufficient to cause the results.JSON.txt files to include results obtained using gpuowl (as well as those from mprime) without my manual intervention going forward?

[kriesel]

I am not aware of a scenario where mprime or prime95 will do automatic results reporting for gpuowl results.
-pool causes multiple gpuowl instances to use a common worktodo file and a common results.txt file.
https://www.mersenneforum.org/showpo...33&postcount=1
Putting the proof files in the mprime directory before the corresponding result records have been reported will cause the mprime instance to loop on attempting to upload a proof file for which the result must be reported first and has not been, generating an error message at each upload attempt. Manually report the previous results.
Try using the primenet.py that ships with gpuowl for automatic reporting and work requesting and proof uploads.

[Reed_Young]

Originally Posted by kriesel

I am not aware of a scenario where mprime or prime95 will do automatic results reporting for gpuowl results.
-pool causes multiple gpuowl instances to use a common worktodo file and a common results.txt file.
https://www.mersenneforum.org/showpo...33&postcount=1

I think I read #3 in this post too optimistically.

Originally Posted by kriesel

Putting the proof files in the mprime directory before the corresponding result records have been reported will cause the mprime instance to loop on attempting to upload a proof file for which the result must be reported first and has not been, generating an error message at each upload attempt. Manually report the previous results.
Try using the primenet.py that ships with gpuowl for automatic reporting and work requesting and proof uploads.

I have tried that, and was hoping there was a better solution because so far, it has not done any good.

Traceback (most recent call last):
  File "~/computing/Mersenne_primes/gpuowl/gpuowl2/tools/primenet.py", line 154, in <module>
    handle(folder)
  File "~/computing/Mersenne_primes/gpuowl/gpuowl2/tools/primenet.py", line 145, in handle
    for entry in os.listdir(folder + 'proof'):
FileNotFoundError: [Errno 2] No such file or directory: './proof'

There is a "proof" directory, which I might reference as $PWD/proof when I'm in the containing directory, but no ./proof directory. And even if I do create a hidden folder, it should be called /.proof, not ./proof. Not sure this is actually relevant, or just a quirk in how Python reports errors but on my system, ./proof would be a terminal command to run an executable named "proof," never a folder name nor a file name.

I do know that creating a .proof folder directly under the main gpuowl folder and copying all the ###,###,###-8.proof files into it, then running primenet.py again, did not help. (In fact it is still there, and was when the above error message was generated. I have also tried putting the .proof file inside the tools folder, which contains primenet.py, but primenet.py still failed to create a results.JSON.txt file.)

[moebius]

gpuOwl 6.11.364 windows binary and

gpuOwl 6.11.380 linux binary (Ubuntu) are without this error.

[Reed_Young]

That doesn't seem to be helping yet. The executable is working fine, but only on crunching the "next" exponent. Even after copying the proof folder and the results.txt file for the finished exponents into the new gpuowl directory, no results.JSON.txt file is created.

[kriesel]

. is current directory. ./proof is a subdirectory of the current, named proof.
You seem mistakenly fixated on there being a gpuowl output file named results.json.txt, which prime95 does, but gpuowl does not. IIRC mprime/prime95 is the ONLY GIMPS app that outputs to results.json.txt and results.txt by default. The rest all default to results.txt only.
Gpuowl puts json formatted results output in its own results.txt.
Gpuowl is not prime95. They were written differently, by different authors. So don't expect them to act the same in all details.

[Reed_Young]

...especially when it's information you already know.

Originally Posted by kriesel

You seem mistakenly fixated on there being a gpuowl output file named results.json.txt, which prime95 does, but gpuowl does not. IIRC mprime/prime95 is the ONLY GIMPS app that outputs to results.json.txt and results.txt by default. The rest all default to results.txt only.
Gpuowl puts json formatted results output in its own results.txt.

Oh, I see. Applying that information has solved the problem. Thanks.

Both of the 2 pages for upload of manual testing results I have encountered could be clearer about that. They don't contradict the facts you just explained, but they also say nothing about what the correct file to upload is for users of software other than mprime/prime95. The one where the Results link under Manual Testing on mersenne.org points is particularly emphatic about not uploading "results.txt" files. Maybe this is obvious to users of related software other than mprime and gpuowl, but that set does not include me so there was really no way I could have known that until somebody explained what you explained. Thanks again!

[kriesel]

The whole problem could have been avoided by reading the gpuowl program's documentation before trying to run it.
It frequently surprises me how imaginative people are in their misconceptions, and how resistant to multiple clues that contradict them. In this case:
1) program behavior persistently not matching, despite multiple attempts to make it match, the misconception-based errant expectations from overgeneralizing prime95's behavior to other programs;
2) nearly every gpuowl build for Windows I've posted includes a help.txt file, generated by gpuowl-win -h >help.txt. Quoting the relevant portions of v6.11-382's help output:

-dir <folder>      : specify local work directory (containing worktodo.txt, results.txt, config.txt, gpuowl.log)
-pool <dir>        : specify a directory with the shared (pooled) worktodo.txt and results.txt
                     Multiple GpuOwl instances, each in its own directory, can share a pool of assignments and report
                     the results back to the common pool.
-results <file>    : name of results file, default 'results.txt'

Note how that does NOT say results.json.txt.
And does not contain the string "json" anywhere.
3) And for anyone who builds it themselves, gpuowl -h or gpuowl-win -h (depending on OS) will output the help content programmed into the gpuowl application.
4) gpuowl prebuilt or compiled by git download will also include readme.md. Which contains no occurrences of "json", but does contain the following:

## Files used by gpuOwl
* `worktodo.txt` : contains exponents to test, one entry per line
* `results.txt` : contains the results
* `N.owl` : the most recent checkpoint for exponent <N>; will resume from here
* `N-prev.owl` : the previous checkpoint, to be used if N.ll is lost or corrupted
* `N.iteration.owl` : a persistent checkpoint at the given iteration

## `worktodo.txt`
The lines in `worktodo.txt` must be of one of these forms:
* `70100200`
* `PRP=FCECE568118E4626AB85ED36A9CC8D4F,1,2,77936867,-1,75,0`

5) When all else fails, the source code is also available for reading.
6) Those too impatient to read other documentation could follow the step by step install and run process given in post one of https://www.mersenneforum.org/showthread.php?t=23391, which specifies looking to "results.txt" output from gpuowl.

https://www.mersenne.org/proof_upload_manual/ makes references to "JSON results" but not to results.json.txt.

https://www.mersenne.org/manual_result/ says "If you are uploading results from Prime95/mprime, please be sure to upload results.json.txt and not results.txt – the JSON-formatted results contain more and better data than the old-style result lines. If your version of Prime95/mprime does not produce results.json.txt, please upgrade to the current version." That clearly applies to prime95 & mprime, neither of which are gpuowl, Mlucas, etc.

The web pages are written with a focus toward prime95/mprime, which is the bulk of the volume and especially of the novice users. GPU application users need to be a bit more savvy and willing to read and apply documentation. And think and troubleshoot logically.

[Andrew Usher]

One can't expect a new user to even know what 'JSON' is or why it might be needed. And indeed the Gpuowl documentation, as you've shown, does not mention producing results in JSON format. So confusion is not unexpected.

[Reed_Young]

Originally Posted by kriesel

The whole problem could have been avoided by reading the gpuowl program's documentation before trying to run it.

No, because that documentation also doesn't explain that the (forbidden, per both web pages) "results.txt" file is JSON-formatted. So your assumption that I did not read that documentation is also unfounded.

Originally Posted by kriesel

-dir <folder>      : specify local work directory (containing worktodo.txt, results.txt, config.txt, gpuowl.log)
-pool <dir>        : specify a directory with the shared (pooled) worktodo.txt and results.txt
                     Multiple GpuOwl instances, each in its own directory, can share a pool of assignments and report
                     the results back to the common pool.
-results <file>    : name of results file, default 'results.txt'

Note how that does NOT say results.json.txt.
And does not contain the string "json" anywhere.

You're getting closer to identifying the problem, but you're still acting like it's the solution. The web pages where the file is to be uploaded both say JSON is required and one specifically says "results.txt" is forbidden, so citing yet another source that says absolutely nothing about JSON one way or the other is a non sequitur.

Clarifying the online documentation was a constructive suggestion. I do not know why it struck a nerve with you but I do know it is not my fault.
https://www.youtube.com/watch?v=7rcb9KI07XE&t=62s

[kriesel]

Originally Posted by Reed_Young

I do know it is not my fault.
https://www.youtube.com/watch?v=7rcb9KI07XE&t=62s

Still refusing to read what was actually written I see, or acknowledge your own very persistent error. (An exceedingly rare one, I don't recall seeing before in YEARS of forum activity.)
The manual result web page is quite clear that the directive toward results.json.txt IS SPECIFIC TO PRIME95/MPRIME.
You had MANY opportunities to detect that the error was yours, and still seem to be denying it.

Here's yet another: read the gpuowl version primenet.py code.

What you are doing is similar to a pilot who somehow miraculously survives controlled flight into terrain because of clinging to his own error instead of reevaluating in time, blaming the aircraft manufacturer, the map maker, air traffic control, anyone other than himself, for his own incorrect interpretation of instrument readings and bad decision making that directly caused a crash. At least in GIMPS the stakes are low.

I have modified https://www.mersenneforum.org/showpo...9&postcount=28.
"Default results file name for most applications is "results.txt". Mprime/prime95 is the outlier, with both results.txt and results.json.txt."
I have also added JSON to the glossary reference info post.
But I doubt that these changes would have stopped you in persisting in your misconception either, after so many other sources of data were willfully misinterpreted or ignored. Ego defense is a stubborn thing.