Feasibility of GPUOwl with Primenet.py having its own computer listing in GIMPS account?
 

I stopped putting it off and set aside enough time today to get Primenet.py working with my GPUOwl installation. It's a very nice tool and not having to haul proof files over to the drive where I have Prime95 running is a definite plus. But, account-wise its work is still checked out and returned under the "manual testing" computer.


I am assuming that making Primenet.py register a discrete computer in your account like an Internet-connected version of Prime95 would do is not trivial, if it has not been done already. Or is there some sort of security difficulty with software other than specifically Prime95 being allowed to do this?


For me the functionality is mainly of interest because manual testing's "smallest exponents" stop at cat 2. My CPU is not really fast enough to qualify for cat 0 first-time PRP and so I set it doing DC, but my GPU in the same system easily is. The manual testing category limit is of course reasonable with the prevalence of patchy internet / sneakernet in that assignment method, but if someone is running GPUOwl with Primenet.py it should be effectively guaranteed that they have the same or better connectivity as an online Prime95 user would.


My apologies if there is already discussion along this line of thought; a search for "primenet computer" turned up nothing relevant.

New PrimeNet script
 

[QUOTE=techn1ciaN;591429]I am assuming that making Primenet.py register a discrete computer in your account like an Internet-connected version of Prime95 would do is not trivial, if it has not been done already.[/QUOTE]

@danc2 and I created a new PrimeNet script last year do this. Please see [URL="https://www.mersenneforum.org/showthread.php?t=26574"]our dedicated thread[/URL] for more information.

It does not yet support uploading PRP proofs, which is why we have not officially announced GpuOwl support. I have implement most of the needed code in my local copy of the script, but I have not yet collected enough PRP proofs from GpuOwl to properly test it, so this will likely be added in the next couple of months after I have finished testing.

[QUOTE=techn1ciaN;591429]For me the functionality is mainly of interest because manual testing's "smallest exponents" stop at cat 2. My CPU is not really fast enough to qualify for cat 0 first-time PRP and so I set it doing DC, but my GPU in the same system easily is. The manual testing category limit is of course reasonable with the prevalence of patchy internet / sneakernet in that assignment method, but if someone is running GPUOwl with Primenet.py it should be effectively guaranteed that they have the same or better connectivity as an online Prime95 user would.[/QUOTE]

Yes, allowing the program to get much smaller category 0 and 1 exponents is one of the main advantages of our PrimeNet script, along with assignment progress reporting.

[QUOTE=tdulcet;591663]@danc2 and I created a new PrimeNet script last year do this. Please see [URL="https://www.mersenneforum.org/showthread.php?t=26574"]our dedicated thread[/URL] for more information.[/QUOTE]


Thank you — this is exactly the idea! I will look into setting it up right now.


[QUOTE=tdulcet;591663]It does not yet support uploading PRP proofs, which is why we have not officially announced GpuOwl support.[/QUOTE]


That should not be a problem; I can just go back to using my Prime95 instance as an uploader for the time being. I think current GPUOwl PRP lines have a flag that a proof was generated, so the server will not assign meaningless TF or DC before you can get the proof uploaded.


[QUOTE=tdulcet;591663]I have implement most of the needed code in my local copy of the script, but I have not yet collected enough PRP proofs from GpuOwl to properly test it, so this will likely be added in the next couple of months after I have finished testing.[/QUOTE]


Would it be at all helpful to you if I were to send along my own proofs (after the corresponding CERT finished, of course) along with whatever other files are relevant? I complete a wavefront test about every three days on a Radeon 5700 XT, using proof power 10.

Extremely cool — thanks again!


You mentioned several months ago in your own thread that you would be open to adding TF functionality if someone else wrote the code for it. Did anything ever come of that? Obviously there wouldn't be an assignment-availability advantage as with GPUOwl*, but I found configuring Misfit for just one MFakt[x] instance a bit obtuse, since much of its functionality (the inbound and outbound "staging") is tuned specifically for the use case of operating many different instances and becomes bloat if you are not doing this.


* That being said, there is a specific manual assignments sub-page for Misfit and anecdotally it has been reserving noticeably smaller exponents than the user-facing manual GPU assignment tool was. I suppose this is an unofficial / workaround way of accomplishing the same thing for GPU TF as the official assignment rules accomplish for primality testing.

[QUOTE=techn1ciaN;591728]Thank you — this is exactly the idea! I will look into setting it up right now.[/QUOTE]

No problem, I am happy it is useful for you. Thanks for helping test it. There is more information about the GpuOwl support in [URL="https://www.mersenneforum.org/showpost.php?p=587064&postcount=5"]this post[/URL] in a different thread. From your screenshot, it looks like you were able to successfully set it up. Feedback is welcome.

[QUOTE=techn1ciaN;591728]I think current GPUOwl PRP lines have a flag that a proof was generated, so the server will not assign meaningless TF or DC before you can get the proof uploaded.[/QUOTE]

Yes, that is correct.

[QUOTE=techn1ciaN;591728]Would it be at all helpful to you if I were to send along my own proofs (after the corresponding CERT finished, of course) along with whatever other files are relevant? I complete a wavefront test about every three days on a Radeon 5700 XT, using proof power 10.[/QUOTE]

Unfortunately not after they have already been uploaded, as they can only be uploaded to the PrimeNet server once.

I currently only have one system that I can run GpuOwl on and it has an extremely slow Intel UHD Graphics P630 GPU, which takes over 80 days per wavefront test. The [URL="https://www.mersenne.org/report_exponent/?exp_lo=111655073&full=1"]current assignment[/URL] has around 19 days to go, at which point I will be able to finish, test and publish my implementation. Although, I will probably wait until at least one more test completes before officially announcing GpuOwl support.

[QUOTE=techn1ciaN;591737]You mentioned several months ago in your own thread that you would be open to adding TF functionality if someone else wrote the code for it. Did anything ever come of that? Obviously there wouldn't be an assignment-availability advantage as with GPUOwl*, but I found configuring Misfit for just one MFakt[x] instance a bit obtuse, since much of its functionality (the inbound and outbound "staging") is tuned specifically for the use case of operating many different instances and becomes bloat if you are not doing this.


* That being said, there is a specific manual assignments sub-page for Misfit and anecdotally it has been reserving noticeably smaller exponents than the user-facing manual GPU assignment tool was. I suppose this is an unofficial / workaround way of accomplishing the same thing for GPU TF as the official assignment rules accomplish for primality testing.[/QUOTE]

No, no one has added TF support to our PrimeNet script yet. The difference of our PrimeNet script is that it uses the [URL="http://v5.mersenne.org/v5design/v5webAPI_0.97.html"]PrimeNet API[/URL] like Prime95/MPrime instead of wrapping the [URL="https://www.mersenne.org/manual_assignment/"]manual assignment pages[/URL], but as you said, that does not provide many advantages for TF assignments. Someone else on this forum can correct me, but I believe most users actually get their TF assignments from the [URL="https://www.gpu72.com/"]GPU72 project[/URL], which has a separate API. The GPU72 project reserves large blocks of TF assignments in advanced, which is why users are able to get smaller exponents from them. Many other PrimeNet scripts do support TF assignments, either by using the GPU72 API or by wrapping the [URL="https://www.mersenne.org/manual_gpu_assignment/"]manual GPU assignment page[/URL].

[QUOTE=tdulcet;591757]Unfortunately not after they have already been uploaded, as they can only be uploaded to the PrimeNet server once.

I currently only have one system that I can run GpuOwl on and it has an extremely slow Intel UHD Graphics P630 GPU, which takes over 80 days per wavefront test.[/QUOTE]

Yes, I realized after I made the post that my offer was a bit misguided. :innocent: I don't believe I would have a problem sending you un-uploaded proofs, but I think PrimeNet may reject a proof from a different user than the one who did the test anyway; I'm not sure.

I am frankly surprised you were able to get primality testing running on an IGP at all. I attempted to set the IGP in my laptop's Ryzen 4600H doing DC-via-PRP, but GPUOwl threw GEC failures like nuts (20–30 within 1M iters.), so I had to conclude that AMD probably did not accommodate for GPU double-precision arithmetic. I loaded an old version of MFakto instead and now get a bonus GPU TF assignment done every couple days — better than nothing.

[QUOTE=tdulcet;591757]Someone else on this forum can correct me, but I believe most users actually get their TF assignments from the [URL="https://www.gpu72.com/"]GPU72 project[/URL], which has a separate API. The GPU72 project reserves large blocks of TF assignments in advanced, which is why users are able to get smaller exponents from them. Many other PrimeNet scripts do support TF assignments, either by using the GPU72 API or by wrapping the [URL="https://www.mersenne.org/manual_gpu_assignment/"]manual GPU assignment page[/URL].[/QUOTE]

I should have been clearer; I am using such a tool — Misfit — presently (I just find that it has so much functionality as to be difficult to configure for a simpler use case). The GPU TF exponents I had been reserving myself were in the 117,xxx,xxx range, but the ones Misfit reserves for me are presently 108,xxx,xxx, even though it does not use the API and I did not configure it for GPU72. I forget how I noticed, but it does contact a specific, purpose-made sub-page of the GPU assignments tool (something like [...]/manual_gpu_assignment/misfit.php), so I figure this has to be the reason why.

GPU72 is cool, but I try to avoid accumulating Internet accounts where possible, so I am fine to get GPU TF from GIMPS directly. I run my TF on my laptop anyway, so it would probably not be best for me to check out the absolute smallest exponents available, with the possibility of occasionally being away from reliable mains power.

[QUOTE=techn1ciaN;591833]I don't believe I would have a problem sending you un-uploaded proofs, but I think PrimeNet may reject a proof from a different user than the one who did the test anyway; I'm not sure.[/QUOTE]

Thanks for the offer! I could upload your PRP proofs, but yes, I would need your PrimeNet User ID. If that is OK with you, feel free to upload the proofs to your favorite file sharing service and then send me a PM with the link and your User ID. I would only need 3-4 proofs to thoroughly test my implementation.

[QUOTE=techn1ciaN;591833]I am frankly surprised you were able to get primality testing running on an IGP at all. I attempted to set the IGP in my laptop's Ryzen 4600H doing DC-via-PRP, but GPUOwl threw GEC failures like nuts (20–30 within 1M iters.), so I had to conclude that AMD probably did not accommodate for GPU double-precision arithmetic. I loaded an old version of MFakto instead and now get a bonus GPU TF assignment done every couple days — better than nothing.[/QUOTE]

That many GEC failures is concerning and it may effect the trial factoring reliability as well. I have gotten GpuOwl to successfully run on a couple of Intel integrated GPUs. The main problem on this system is that sometimes GpuOwl hangs after it is started, but I believe this is a driver issue.

[QUOTE=techn1ciaN;591833]I should have been clearer; I am using such a tool — Misfit — presently (I just find that it has so much functionality as to be difficult to configure for a simpler use case). The GPU TF exponents I had been reserving myself were in the 117,xxx,xxx range, but the ones Misfit reserves for me are presently 108,xxx,xxx, even though it does not use the API and I did not configure it for GPU72. I forget how I noticed, but it does contact a specific, purpose-made sub-page of the GPU assignments tool (something like [...]/manual_gpu_assignment/misfit.php), so I figure this has to be the reason why.[/QUOTE]

All my systems that I have contributed to GIMPS on are Linux and I personally have no interest in trial factoring, so I have never used Misfit. However, that is interesting that it has its own separate page/endpoint, which provides much smaller exponents. There would then be four ways for users to get TF assignments:
[LIST][*]The PrimeNet API (like Prime95/MPrime)[*]The manual GPU assignment page[*]The Misfit specific manual GPU assignment page[*]The GPU72 API[/LIST]From your explanation, it seems that Misfit supports the last two methods. As I said, there are other PrimeNet scripts that support the second and last methods, notably this one by @teknohog: [URL]https://github.com/teknohog/primetools[/URL]. If the Misfit specific page supports smaller exponents, it would likely be trivial to update that script to use it instead. I would of course need to look at the Misfit source code to know more...

[QUOTE=tdulcet;591852]I would only need 3-4 proofs to thoroughly test my implementation.[/QUOTE]

To be clear — am I fine to send along proofs as I finish the tests, or do you prefer that I collect a batch of 3 or 4 and send them all at once? I would find the first case easier but am fine to wait if there is a good reason.

[QUOTE=tdulcet;591852]That many GEC failures is concerning and it may effect the trial factoring reliability as well. I have gotten GpuOwl to successfully run on a couple of Intel integrated GPUs.[/QUOTE]

I do not believe that there actually is a system reliability problem.

The reason why TF is so fast on a GPU is that it can be executed with single-precision arithmetic and most consumer GPUs have single-precision throughput that is a large multiple of the double-precision value (1:16 seems standard for AMD and 1:32 for Nvidia). Everything else GIMPS does (PRP, P-/+1, LL, I believe even ECM) requires double-precision.

I do not have the specific thread, but I can recall reading on here that some IGPs simply do not implement double-precision capability at all (and for most users this would of course not be a great loss). One then expects that a program dependent upon double-precision operations would either crash or give results very different from those expected. GPUOwl with GEC seems to be the latter case.

My laptop is recent and well-maintained so I find it hard to believe that there would be so many GEC failures unless the necessary arithmetic was unsupported altogether. Particularly, Prime95 with GEC on the CPU portion of the same processor throws no failures at all, even when MFakto is running on the IGP portion at the same time. One expects that if there were any significant memory, thermals, or power delivery problem, then Prime95 would be throwing at least some failures before too long.

[QUOTE=techn1ciaN;591881]To be clear — am I fine to send along proofs as I finish the tests, or do you prefer that I collect a batch of 3 or 4 and send them all at once? I would find the first case easier but am fine to wait if there is a good reason.[/QUOTE]

The first case is fine with me, if that would be easier for you. Thanks again for doing this! It will significantly speed up the time it takes to test my proof uploading implementation so I can release it much sooner.

[QUOTE=techn1ciaN;591881]I do not have the specific thread, but I can recall reading on here that some IGPs simply do not implement double-precision capability at all (and for most users this would of course not be a great loss). One then expects that a program dependent upon double-precision operations would either crash or give results very different from those expected. GPUOwl with GEC seems to be the latter case.[/QUOTE]

Hmm, many people have ran GpuOwl on integrated GPUs, you can see lot of benchmarks on the main GpuOwl thread. If the needed operations were not supported, I would expect the OpenCL code to not even compile or run at all. If the GPU hardware does not support double-precision floating point, it could be using a software based implementation, but that would of course significantly impact performance. I would encourage you post about this on the main thread where someone should be able to provide more insight.

[QUOTE=techn1ciaN;591881]My laptop is recent and well-maintained so I find it hard to believe that there would be so many GEC failures unless the necessary arithmetic was unsupported altogether. Particularly, Prime95 with GEC on the CPU portion of the same processor throws no failures at all, even when MFakto is running on the IGP portion at the same time. One expects that if there were any significant memory, thermals, or power delivery problem, then Prime95 would be throwing at least some failures before too long.[/QUOTE]

The problem could be with the GPU itself or the GPU RAM if it has dedicated RAM. If so, you could run some [URL="https://www.pcworld.com/article/456919/keep-it-stable-stupid-how-to-stress-test-your-pc-hardware.html"]stress/torture tests[/URL] to potentially narrow down the issue.

[QUOTE=tdulcet;591967]The first case is fine with me, if that would be easier for you. Thanks again for doing this![/QUOTE]

Great. No problem; happy to help. My next GPUOwl test should finish in a bit less than 2½ days (although I plan to upgrade the relevant machine to Windows 11 later this evening, so that could go up significantly if there is some Microsoft idiosyncrasy that tanks my iters/s).

[QUOTE=tdulcet;591697]The problem could be with the GPU itself or the GPU RAM if it has dedicated RAM. If so, you could run some [URL="https://www.pcworld.com/article/456919/keep-it-stable-stupid-how-to-stress-test-your-pc-hardware.html"]stress/torture tests[/URL] to potentially narrow down the issue.[/QUOTE]

I performed a simple matching double-check by loading

[CODE]Factor=[M]9040249[/M],70,71[/CODE]and verifying that MFakto was able to find the correct factor. I will run this "test" one more time and if it passes again, then I will conclude that my IGP is stable enough for trial factoring under my running conditions and leave the matter there. I am probably not interested in running GPUOwl anymore even if it were hypothetically possible to resolve the GEC failures, since with Prime95 running concurrently on the CPU, a single wavefront DC would take me nearly a month at 100% uptime. (By contrast, MFakto is able to complete an FTC wavefront TF every couple of days.)

[QUOTE=techn1ciaN;592006]My next GPUOwl test should finish in a bit less than 2½ days (although I plan to upgrade the relevant machine to Windows 11 later this evening, so that could go up significantly if there is some Microsoft idiosyncrasy that tanks my iters/s).[/QUOTE]

Upgrade finished. My throughput is the same, so we should be on schedule. :thumbs-up: (Unless my new GPU undervolt that I validated as stable in 10 is somehow not so in 11. I burned it in for 24 hours with GEC every 10,000 iters., so I hope that it would not be so close to the edge of reliable that this would be a possibility, but I will still review the printout for any GEC failures when I wake up next morning.)

[QUOTE=techn1ciaN;592006]I will run this "test" one more time and if it passes again, then I will conclude that my IGP is stable enough for trial factoring under my running conditions and leave the matter there.[/QUOTE]

My second "double-check" of [CODE]Factor=[M]228288397[/M],73,74[/CODE] also validated, so if I do have any intrinsic hardware issue, it is strictly limited to double precision–dependent operations and I am safe to run TF. Case closed. :thumbs-up:

[QUOTE=tdulcet;591757]Thanks for helping test it.[/QUOTE]

What is the oldest GPUOwl that your script is intended to work with? In spite of my earlier comments, I did end up revisiting the IGP issue and seem to have been able to fix the problem by loading a slightly older GPUOwl* (6.11 vs. 7.2). I registered the GPUOwl instance with your script, but the functionality does not seem to be working very well; there is no progress reporting at all, which has the side effect of the "days_work" parameter not working properly (I set it to 1 but a second assignment was still fetched even though the current one has ~12 days to go). Also, even though an assignment was successfully retrieved under the appropriate computer, the assignment I already had in the GPUOwl WorkToDo failed to associate itself and is still listed under "Manual Testing" in my account.

In my [URL]https://www.mersenne.org/cpus/[/URL], the new computer says that it is GPUOwl v7.2, even though the actual running version is 6.11 as stated. I don't know how relevant that is.

* Maybe. I also changed a few other variables at the same time and can't say for certain which was the deciding one.

[QUOTE=techn1ciaN;592406]What is the oldest GPUOwl that your script is intended to work with?[/QUOTE]

I specifically targeted and tested with GpuOwl v6.11-380 and the latest version (currently v7.2-70), although it likely also works with the versions in between.

[QUOTE=techn1ciaN;592406]I registered the GPUOwl instance with your script, but the functionality does not seem to be working very well; there is no progress reporting at all, which has the side effect of the "days_work" parameter not working properly (I set it to 1 but a second assignment was still fetched even though the current one has ~12 days to go). Also, even though an assignment was successfully retrieved under the appropriate computer, the assignment I already had in the GPUOwl WorkToDo failed to associate itself and is still listed under "Manual Testing" in my account.[/QUOTE]

Hmm, something is not right... What worktype are you doing? Would you mind sharing the output from the script when run with the [C]-d/--debug[/C] option (i.e. the output from this command: [C]python3 primenet.py -d -t 0[/C]), as well as the last 10-20 lines or so of your [C]gpuowl.log[/C] file.

[QUOTE=techn1ciaN;592406]In my [URL]https://www.mersenne.org/cpus/[/URL], the new computer says that it is GPUOwl v7.2, even though the actual running version is 6.11 as stated. I don't know how relevant that is.[/QUOTE]

Yes, there is no way for the script to know which version of GpuOwl the user is actually using, so it just reports the latest version in the application version string when registering with PrimeNet. You could manually modify the [C]programs[/C] dictionary in the script (see [URL="https://github.com/tdulcet/Distributed-Computing-Scripts/blob/fd73c1de915e1f573212491ebec4c7e62f9ebfd9/primenet.py#L331-L332"]here[/URL]) to set a different version.

[QUOTE=tdulcet;592414]Would you mind sharing the output from the script when run with the [C]-d/--debug[/C] option (i.e. the output from this command: [C]python3 primenet.py -d -t 0[/C]), as well as the last 10-20 lines or so of your [C]gpuowl.log[/C] file.[/QUOTE]

I'm apparently not incredibly sharp. :innocent: I forgot that the script's default name for the work file is worktodo.ini, and hadn't changed that for this instance. The exponent I had checked out manually was in worktodo.txt, so upon first run the script looked for worktodo.ini, found nothing, created that file, then checked the file, found no work, and added some. There was no progress reporting because the exponent actually being worked on was in a file the script didn't know about.

That being said, I think it may be a good idea for you to change your default work file name. Current Prime95, current GPUOwl, and "current" (most recent) CUDALucas all use worktodo.txt and not .ini.

[QUOTE=techn1ciaN;592443]I'm apparently not incredibly sharp. :innocent: I forgot that the script's default name for the work file is worktodo.ini, and hadn't changed that for this instance. The exponent I had checked out manually was in worktodo.txt, so upon first run the script looked for worktodo.ini, found nothing, created that file, then checked the file, found no work, and added some. There was no progress reporting because the exponent actually being worked on was in a file the script didn't know about.[/QUOTE]

I am glad you figured it out. I guess I should have mentioned that gotcha regarding the worktodo file. I use a wrapper script to start GpuOwl (see [URL="https://www.mersenneforum.org/showpost.php?p=573177&postcount=44"]here[/URL]), so this is not an issue for me.

[QUOTE=techn1ciaN;592443]That being said, I think it may be a good idea for you to change your default work file name. Current Prime95, current GPUOwl, and "current" (most recent) CUDALucas all use worktodo.txt and not .ini.[/QUOTE]

Well, our PrimeNet script does not support Prime95/MPrime, as it has PrimeNet functionality built in, but I take your point. Our script is adapted from Loïc Le Loarer's script for Mlucas, which is why it uses [C]worktodo.ini[/C]. If I changed the default filename to [C]worktodo.txt[/C], it would then cause issues for those users. Our script is also going to be the recommend one by @ewmayer starting with his next version of Mlucas which [URL="https://www.mersenneforum.org/showpost.php?p=584565&postcount=6"]supports PRP proofs[/URL].

Anyway, I think the best solution would be for the GIMPS program authors to agree to a standard for the filenames. If you read the post linked at the top of my post #5 above, you will see that there is another similar issue where GpuOwl does not follow a convention set by Prime95/MPrime and Mlucas...

[QUOTE=tdulcet;591757]Feedback is welcome.[/QUOTE]

Are you currently taking feature requests?

One might receive a first-time PRP assignment with no P-1 done. This can mess with the computation of how much work is queued, since that goes off the assumption that the full-length PRP test will always happen, even if the possibility of a P-1 factor makes that not necessarily true. So especially with days_work values low enough to qualify for low-category exponents, one might finish a PRP test, clear their next assignment more quickly than expected by finding a P-1 factor, and be left with no work in the file. This is a particular problem for GPUOwl since the program terminates in this case, so the ability to "set and forget" when you are not going to have consistent access to the client computer (e.g., going on a trip) is lessened or eliminated.

Prime95 has a useful option that can avoid this by, if an assignment comes in with no P-1 done, doing the P-1 immediately before continuing the current full-length test (SequentialWorkToDo in undoc.txt). Is this something that would be feasible for your script to implement?

Alternatively, I can think of a simpler way to solve the same problem: When checking the work file to see if more work needs to be fetched, if a PRP line indicates that P-1 is still needed AND it is the only line present besides the active exponent AND days_work (as opposed to num_cache) is being used, then that line will be ignored. This would ensure that there is always at least one assignment guaranteed to be full-length OR at least two assignments if the next assignment needs P-1.

[QUOTE=techn1ciaN;592589]Are you currently taking feature requests?[/QUOTE]

Yes, although no promises that anything will be implemented. There is a list of features at the bottom of my README (see [URL="https://github.com/tdulcet/Distributed-Computing-Scripts#contributing"]here[/URL]) that I would like to implement at some point. Most likely, anything requested will be added to that list, although it would help prioritize what I work on next after the PRP proof uploading. Pull requests are of course welcome!

[QUOTE=techn1ciaN;592589]One might receive a first-time PRP assignment with no P-1 done. This can mess with the computation of how much work is queued, since that goes off the assumption that the full-length PRP test will always happen, even if the possibility of a P-1 factor makes that not necessarily true. So especially with days_work values low enough to qualify for low-category exponents, one might finish a PRP test, clear their next assignment more quickly than expected by finding a P-1 factor, and be left with no work in the file. This is a particular problem for GPUOwl since the program terminates in this case, so the ability to "set and forget" when you are not going to have consistent access to the client computer (e.g., going on a trip) is lessened or eliminated.[/QUOTE]

Yes, this is a known issue with our PrimeNet script. However, note that users only have around a 1 in 20 or less chance of finding a factor and most of the lower category 0 and 1 assignments already have P-1 done, so your chances of this happening is very low.

[QUOTE=techn1ciaN;592589]Prime95 has a useful option that can avoid this by, if an assignment comes in with no P-1 done, doing the P-1 immediately before continuing the current full-length test (SequentialWorkToDo in undoc.txt). Is this something that would be feasible for your script to implement?[/QUOTE]

Interesting, I was aware of that option, but I did not know this is what it was for. I think it would be a feature that the GIMPS programs (i.e. Mlucas and GpuOwl) would need to implement. Our PrimeNet script should already support the assignments being done out of order.

[QUOTE=techn1ciaN;592589]Alternatively, I can think of a simpler way to solve the same problem: When checking the work file to see if more work needs to be fetched, if a PRP line indicates that P-1 is still needed AND it is the only line present besides the active exponent AND days_work (as opposed to num_cache) is being used, then that line will be ignored. This would ensure that there is always at least one assignment guaranteed to be full-length OR at least two assignments if the next assignment needs P-1.[/QUOTE]

Hmm, the reason our PrimeNet script (like Prime95/MPrime) does not queue up two assignments by default is because the [URL="https://www.mersenne.org/thresholds/"]assignment rules[/URL] stipulate that the system must start assignments within 3 days and finish them within 15 days to receive first time category 0 exponents. Assignments can also be recycled if they are not started within 7 days. Queue up two assignments would then preclude all but the fastest GPUs from getting category 0 exponents and it would relegate most systems to a higher category than they would otherwise be eligible for, defeating one of the primary purposes of our script.

However, in your case you said you "complete a wavefront test about every three days", so if that means less than or equal to 3 days, you could likely set the [C]--num_cache 1[/C] option and still be eligible for category 0 exponents. The [C]num_cache[/C] value does not include the current/active assignment, so a [C]num_cache[/C] of 1 would always give you at least two assignments, the same as all the previous PrimeNet scripts do by default. I consider the [C]-n/--num_cache[/C] option deprecated in favor of [C]-W/--days_work[/C] and I do not recommend people use it, but you may be one of the lucky few who would be OK to use it.

Anyway, I am open to solutions to solve for everyone, without preventing any systems from getting lower category assignments. As you probably know, Prime95/MPrime solves by this by directly getting another assignment whenever P-1 finds a factor and it runs out of work, as it has the PrimeNet functionality built in.

[QUOTE=tdulcet;592599]Most likely, anything requested will be added to that list, although it would help prioritize what I work on next after the PRP proof uploading.[/QUOTE]

Work lines without AIDs might be manually loaded into the work file (e.g., someone picking up work from the strategic DC thread). Would it be feasible for your script to register these lines at the next PrimeNet check-in and rewrite them with the AIDs that are returned? Currently, the only way to get AIDs for such lines if you're working them on a GPU is to check them out one by one with the manual assignment tool (tedious; doesn't work for anything that the manual assignment rules prohibit) or to add the lines to your Prime95 work file, let Prime95 reserve and rewrite them, and then move the result (also tedious; can make Prime95 start meaningless P-1 or TF if you use SequentialWorkToDo=2).

Prime95's convention is that adding a work line with no AID will trigger a PrimeNet registration attempt the next time you start the software, unless you write "N/A" where the AID would normally go (to avoid annoying repeat errors with line formats that PrimeNet does not understand yet like Pminus1 / Pplus1.)

I checked your existing to-do list and the only item that seemed like it could be relevant was "Reserve a specific exponent." I don't know whether that refers to this functionality or to something else.

[QUOTE=techn1ciaN;592770]Work lines without AIDs might be manually loaded into the work file (e.g., someone picking up work from the strategic DC thread). Would it be feasible for your script to register these lines at the next PrimeNet check-in and rewrite them with the AIDs that are returned? Currently, the only way to get AIDs for such lines if you're working them on a GPU is to check them out one by one with the manual assignment tool (tedious; doesn't work for anything that the manual assignment rules prohibit) or to add the lines to your Prime95 work file, let Prime95 reserve and rewrite them, and then move the result (also tedious; can make Prime95 start meaningless P-1 or TF if you use SequentialWorkToDo=2).[/QUOTE]

Yes, it would be possible. In my last major update of the PrimeNet script in late August, I implemented about 80% of the code to reserve individual exponents using the [URL="http://v5.mersenne.org/v5design/v5webAPI_0.97.html#ra"]register assignment endpoint[/URL] of the v5 API:
[QUOTE=tdulcet;586990][LIST][*]Implemented many of the necessary changes to eventually support reserving individual assignments using the register assignment endpoint of the v5 API. Added support for parsing assignments in the worktodo file with [C]N/A[/C], [C]n/a[/C] or [C]0[/C] for the assignment ID and assignments with no ID at all.[/LIST][/QUOTE]

I have not finished it, as I am not sure how many people would be interested in manually creating and reserving assignments. I was only interested personally because I wanted our PrimeNet script to support the full functionality of the v5 PrimeNet API. Other than maybe a couple of people on that strategic DC thread, I do not think many people would actually use this functionality. It would also require a refactoring of how the script currently works in order to rewrite the worktodo file and add in the AIDs as you described. This is also why it does not yet support unreserving individual assignments.

Anyway, it does support parsing assignments in the worktodo file without a AID, although it should currently just ignore them. While this is completely unsupported and untested, I believe if you called the [URL="https://github.com/tdulcet/Distributed-Computing-Scripts/blob/master/primenet.py#L117"][C]register_assignment()[/C] function[/URL] in our script from the correct place, it would register your assignments and output the new AIDs, although you would then need to manually copy/paste those AIDs back into your worktodo file. I am not sure if this would be any easier than those two other methods you listed.

[QUOTE=techn1ciaN;592770]Prime95's convention is that adding a work line with no AID will trigger a PrimeNet registration attempt the next time you start the software, unless you write "N/A" where the AID would normally go (to avoid annoying repeat errors with line formats that PrimeNet does not understand yet like Pminus1 / Pplus1.)[/QUOTE]

Yes, it supports parsing assignments following that convention. However, another reason I decided not to finish implementing this feature is that there is no standard for assignments without an AID that is supported by all of the GIMPS programs. From examining the source code for the programs, here is what is specifically supported:
[LIST][*]Prime95/MPrime: [C]N/A[/C], [C]N/a[/C], [C]n/A[/C], [C]n/a[/C] or no AID.[*]Mlucas: No AID if LL and no P-1 needed.[*]CUDALucas: [C]N/A[/C], [C]N/a[/C], [C]n/A[/C], [C]n/a[/C] or no AID.[*]GpuOwl: [C]N/A[/C] or [C]0[/C].[/LIST]As you can see, there is no convention that is supported by all three of the programs our script supports... BTW, our script does support the [C]Pminus1[/C] and [C]PMinus1[/C] (capital M) assignment formats, as those are used by Mlucas.

[QUOTE=techn1ciaN;592770]I checked your existing to-do list and the only item that seemed like it could be relevant was "Reserve a specific exponent." I don't know whether that refers to this functionality or to something else.[/QUOTE]

Yes, that refers to this functionality. It originality said "Support reserving exponents in a range (#4) or a specific exponent", but I completed the first part of that in my last major update. Anyway, I may finish implementing this feature at some point, but in the meantime, anyone is of course welcome to submit a pull request.

[QUOTE=techn1ciaN;592681]I was reluctant to load 6.11 because I used tdulcet's PrimeNet script to register my 5700 XT as a PrimeNet "computer" and have been working towards qualifying it for cat 0 FTCs; I run 7.2 for the fused P-1 since most PRP FTCs seem to come without P-1 done. It occurs to me now, though, that I could simply haul the 6.11 results into the folder with my 7.2 instance and the script, let them be uploaded, and achieve the same result.[/QUOTE]

To respond to your message in the strategic DC thread, I thought I should note that you could put both versions of GpuOwl in the same folder with our PrimeNet script and then run the correct one for whatever worktype you are currently doing. If you want to keep the GpuOwl versions in separate folders, you could also just copy the script ([C]primenet.py[/C]) and then move config file ([C]local.ini[/C]) between the folders when switching versions.

Our PrimeNet script now fully supports GpuOwl
 

To update anyone following this thread, I recently submitted two pull requests to the GpuOwl master/main (see [URL="https://github.com/preda/gpuowl/pull/246"]here[/URL]) and v6 (see [URL="https://github.com/preda/gpuowl/pull/247"]here[/URL]) branches respectively, which @preda merged. They added support for the [C]PRPDC=[/C] prefix for PRP DC assignments, which was needed for GpuOwl to support all of the features of our PrimeNet script. I also took the opportunity to add support for assignments without an assignment ID.

[QUOTE=tdulcet;591663]It does not yet support uploading PRP proofs, which is why we have not officially announced GpuOwl support.[/QUOTE]

Thanks in large part to those PRP proof files provided by @techn1ciaN, I was finally able to finish, test and publish my PRP proof uploading implementation. Our PrimeNet script now fully supports GpuOwl!

It will automatically upload any PRP proof files every hour by default. Like with Prime95/MPrime, this can be disabled by setting a [C]ProofUploads=0[/C] option in the config file. Unlike the script included with GpuOwl, it will automatically delete the proof files after upload. This can be changed by using the [C]--archive_proofs[/C] option to specify a directory to archive the proof files to after upload. It also supports the double-check using PRP with proof (155) worktype. Users can also use the [C]--upload_proofs[/C] option to manually upload any PRP proof files. Run the [C]--help[/C] option for more information.

Compared to the upload script included with GpuOwl, my implementation has these advantages:
[LIST][*]Much more error checking.[*]Only requires reading one 5 MiB chunk into memory at a time, instead of the entire proof file.[*]Only uploads the requested sections of the file, in case it fails and is resumed.[*]Uses HTTPS to upload the proof files.[*]Automatically deletes the proof files after upload by default to save space.[*]Saves the PrimeNet User ID and optional proof archive directory to the config file, so they only need to be provided once.[*]Supports both Python 2 and 3.[*]Proof uploading functionality is ingratiated into the main [C]primenet.py[/C] script, instead of a separate [C]upload.py[/C] script.[/LIST]I also added a new [C]-D/--dir[/C] option to our PrimeNet script, similar to the [C]--dirs[/C] option in the script included with GpuOwl, so that users will only need to run a single copy of the script for multiple instances/workers. It should thus be very easy for users to upgrade.

I would recommend users use our PrimeNet script with the latest version of either the GpuOwl master/main (currently [C]v7.2-91-g9c22195[/C]) or v6 (currently [C]v6.11-382-g98ff9c7[/C]) branches, although it should support all recent versions (unless of course one is doing PRP DC assignments). @ewmayer wrote some great documentation on how to use it on the bottom of [URL="https://www.mersenneforum.org/mayer/README_tdulcet_danc2.html"]this page[/URL]. While his documentation is for Mlucas, most of it also applies to GpuOwl. Just use the [C]-g/--gpuowl[/C] option the first time you run it.

[QUOTE=techn1ciaN;592770]Work lines without AIDs might be manually loaded into the work file (e.g., someone picking up work from the strategic DC thread). Would it be feasible for your script to register these lines at the next PrimeNet check-in and rewrite them with the AIDs that are returned?[/QUOTE]

As you may have seen from [URL="https://www.mersenneforum.org/showthread.php?p=630622#post630622"]my big announcement[/URL] yesterday, our PrimeNet script now fully supports reserving individual assignments. This should work the same as with Prime95/MPrime. Just add one or more assignments without an AID to your worktodo file and the script will automatically attempt to register them. As you described, it will update the assignment with the new AID if successful, otherwise it will just put [C]N/A[/C] for the AID. This supports all worktypes supported by recent versions of GpuOwl, including PRP, LL and P-1 assignments. If GpuOwl and/or the PrimeNet script are already running, you can also now add the assignments to a [C]worktodo.add[/C] file and they will automatically be moved to the end of your regular worktodo file at the next update. I believe this now completes your feature request. Feedback is welcome!