mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > GPU Computing > GpuOwl

Reply
 
Thread Tools
Old 2021-11-04, 13:11   #12
techn1ciaN
 
techn1ciaN's Avatar
 
Oct 2021
U.S. / Maine

89 Posts
Default

Quote:
Originally Posted by tdulcet View Post
Thanks for helping test it.
What is the oldest GPUOwl that your script is intended to work with? In spite of my earlier comments, I did end up revisiting the IGP issue and seem to have been able to fix the problem by loading a slightly older GPUOwl* (6.11 vs. 7.2). I registered the GPUOwl instance with your script, but the functionality does not seem to be working very well; there is no progress reporting at all, which has the side effect of the "days_work" parameter not working properly (I set it to 1 but a second assignment was still fetched even though the current one has ~12 days to go). Also, even though an assignment was successfully retrieved under the appropriate computer, the assignment I already had in the GPUOwl WorkToDo failed to associate itself and is still listed under "Manual Testing" in my account.

In my https://www.mersenne.org/cpus/, the new computer says that it is GPUOwl v7.2, even though the actual running version is 6.11 as stated. I don't know how relevant that is.

* Maybe. I also changed a few other variables at the same time and can't say for certain which was the deciding one.
techn1ciaN is online now   Reply With Quote
Old 2021-11-04, 14:22   #13
tdulcet
 
tdulcet's Avatar
 
"Teal Dulcet"
Jun 2018

2·33 Posts
Default

Quote:
Originally Posted by techn1ciaN View Post
What is the oldest GPUOwl that your script is intended to work with?
I specifically targeted and tested with GpuOwl v6.11-380 and the latest version (currently v7.2-70), although it likely also works with the versions in between.

Quote:
Originally Posted by techn1ciaN View Post
I registered the GPUOwl instance with your script, but the functionality does not seem to be working very well; there is no progress reporting at all, which has the side effect of the "days_work" parameter not working properly (I set it to 1 but a second assignment was still fetched even though the current one has ~12 days to go). Also, even though an assignment was successfully retrieved under the appropriate computer, the assignment I already had in the GPUOwl WorkToDo failed to associate itself and is still listed under "Manual Testing" in my account.
Hmm, something is not right... What worktype are you doing? Would you mind sharing the output from the script when run with the -d/--debug option (i.e. the output from this command: python3 primenet.py -d -t 0), as well as the last 10-20 lines or so of your gpuowl.log file.

Quote:
Originally Posted by techn1ciaN View Post
In my https://www.mersenne.org/cpus/, the new computer says that it is GPUOwl v7.2, even though the actual running version is 6.11 as stated. I don't know how relevant that is.
Yes, there is no way for the script to know which version of GpuOwl the user is actually using, so it just reports the latest version in the application version string when registering with PrimeNet. You could manually modify the programs dictionary in the script (see here) to set a different version.
tdulcet is offline   Reply With Quote
Old 2021-11-04, 19:18   #14
techn1ciaN
 
techn1ciaN's Avatar
 
Oct 2021
U.S. / Maine

89 Posts
Default

Quote:
Originally Posted by tdulcet View Post
Would you mind sharing the output from the script when run with the -d/--debug option (i.e. the output from this command: python3 primenet.py -d -t 0), as well as the last 10-20 lines or so of your gpuowl.log file.
I'm apparently not incredibly sharp. I forgot that the script's default name for the work file is worktodo.ini, and hadn't changed that for this instance. The exponent I had checked out manually was in worktodo.txt, so upon first run the script looked for worktodo.ini, found nothing, created that file, then checked the file, found no work, and added some. There was no progress reporting because the exponent actually being worked on was in a file the script didn't know about.

That being said, I think it may be a good idea for you to change your default work file name. Current Prime95, current GPUOwl, and "current" (most recent) CUDALucas all use worktodo.txt and not .ini.

Last fiddled with by techn1ciaN on 2021-11-04 at 19:22 Reason: Clarification
techn1ciaN is online now   Reply With Quote
Old 2021-11-05, 09:56   #15
tdulcet
 
tdulcet's Avatar
 
"Teal Dulcet"
Jun 2018

2×33 Posts
Default

Quote:
Originally Posted by techn1ciaN View Post
I'm apparently not incredibly sharp. I forgot that the script's default name for the work file is worktodo.ini, and hadn't changed that for this instance. The exponent I had checked out manually was in worktodo.txt, so upon first run the script looked for worktodo.ini, found nothing, created that file, then checked the file, found no work, and added some. There was no progress reporting because the exponent actually being worked on was in a file the script didn't know about.
I am glad you figured it out. I guess I should have mentioned that gotcha regarding the worktodo file. I use a wrapper script to start GpuOwl (see here), so this is not an issue for me.

Quote:
Originally Posted by techn1ciaN View Post
That being said, I think it may be a good idea for you to change your default work file name. Current Prime95, current GPUOwl, and "current" (most recent) CUDALucas all use worktodo.txt and not .ini.
Well, our PrimeNet script does not support Prime95/MPrime, as it has PrimeNet functionality built in, but I take your point. Our script is adapted from Loïc Le Loarer's script for Mlucas, which is why it uses worktodo.ini. If I changed the default filename to worktodo.txt, it would then cause issues for those users. Our script is also going to be the recommend one by @ewmayer starting with his next version of Mlucas which supports PRP proofs.

Anyway, I think the best solution would be for the GIMPS program authors to agree to a standard for the filenames. If you read the post linked at the top of my post #5 above, you will see that there is another similar issue where GpuOwl does not follow a convention set by Prime95/MPrime and Mlucas...
tdulcet is offline   Reply With Quote
Old 2021-11-06, 01:26   #16
techn1ciaN
 
techn1ciaN's Avatar
 
Oct 2021
U.S. / Maine

89 Posts
Default

Quote:
Originally Posted by tdulcet View Post
Feedback is welcome.
Are you currently taking feature requests?

One might receive a first-time PRP assignment with no P-1 done. This can mess with the computation of how much work is queued, since that goes off the assumption that the full-length PRP test will always happen, even if the possibility of a P-1 factor makes that not necessarily true. So especially with days_work values low enough to qualify for low-category exponents, one might finish a PRP test, clear their next assignment more quickly than expected by finding a P-1 factor, and be left with no work in the file. This is a particular problem for GPUOwl since the program terminates in this case, so the ability to "set and forget" when you are not going to have consistent access to the client computer (e.g., going on a trip) is lessened or eliminated.

Prime95 has a useful option that can avoid this by, if an assignment comes in with no P-1 done, doing the P-1 immediately before continuing the current full-length test (SequentialWorkToDo in undoc.txt). Is this something that would be feasible for your script to implement?

Alternatively, I can think of a simpler way to solve the same problem: When checking the work file to see if more work needs to be fetched, if a PRP line indicates that P-1 is still needed AND it is the only line present besides the active exponent AND days_work (as opposed to num_cache) is being used, then that line will be ignored. This would ensure that there is always at least one assignment guaranteed to be full-length OR at least two assignments if the next assignment needs P-1.

Last fiddled with by techn1ciaN on 2021-11-06 at 01:28 Reason: Corrected faulty condition
techn1ciaN is online now   Reply With Quote
Old 2021-11-06, 12:05   #17
tdulcet
 
tdulcet's Avatar
 
"Teal Dulcet"
Jun 2018

2·33 Posts
Default

Quote:
Originally Posted by techn1ciaN View Post
Are you currently taking feature requests?
Yes, although no promises that anything will be implemented. There is a list of features at the bottom of my README (see here) that I would like to implement at some point. Most likely, anything requested will be added to that list, although it would help prioritize what I work on next after the PRP proof uploading. Pull requests are of course welcome!

Quote:
Originally Posted by techn1ciaN View Post
One might receive a first-time PRP assignment with no P-1 done. This can mess with the computation of how much work is queued, since that goes off the assumption that the full-length PRP test will always happen, even if the possibility of a P-1 factor makes that not necessarily true. So especially with days_work values low enough to qualify for low-category exponents, one might finish a PRP test, clear their next assignment more quickly than expected by finding a P-1 factor, and be left with no work in the file. This is a particular problem for GPUOwl since the program terminates in this case, so the ability to "set and forget" when you are not going to have consistent access to the client computer (e.g., going on a trip) is lessened or eliminated.
Yes, this is a known issue with our PrimeNet script. However, note that users only have around a 1 in 20 or less chance of finding a factor and most of the lower category 0 and 1 assignments already have P-1 done, so your chances of this happening is very low.

Quote:
Originally Posted by techn1ciaN View Post
Prime95 has a useful option that can avoid this by, if an assignment comes in with no P-1 done, doing the P-1 immediately before continuing the current full-length test (SequentialWorkToDo in undoc.txt). Is this something that would be feasible for your script to implement?
Interesting, I was aware of that option, but I did not know this is what it was for. I think it would be a feature that the GIMPS programs (i.e. Mlucas and GpuOwl) would need to implement. Our PrimeNet script should already support the assignments being done out of order.

Quote:
Originally Posted by techn1ciaN View Post
Alternatively, I can think of a simpler way to solve the same problem: When checking the work file to see if more work needs to be fetched, if a PRP line indicates that P-1 is still needed AND it is the only line present besides the active exponent AND days_work (as opposed to num_cache) is being used, then that line will be ignored. This would ensure that there is always at least one assignment guaranteed to be full-length OR at least two assignments if the next assignment needs P-1.
Hmm, the reason our PrimeNet script (like Prime95/MPrime) does not queue up two assignments by default is because the assignment rules stipulate that the system must start assignments within 3 days and finish them within 15 days to receive first time category 0 exponents. Assignments can also be recycled if they are not started within 7 days. Queue up two assignments would then preclude all but the fastest GPUs from getting category 0 exponents and it would relegate most systems to a higher category than they would otherwise be eligible for, defeating one of the primary purposes of our script.

However, in your case you said you "complete a wavefront test about every three days", so if that means less than or equal to 3 days, you could likely set the --num_cache 1 option and still be eligible for category 0 exponents. The num_cache value does not include the current/active assignment, so a num_cache of 1 would always give you at least two assignments, the same as all the previous PrimeNet scripts do by default. I consider the -n/--num_cache option deprecated in favor of -W/--days_work and I do not recommend people use it, but you may be one of the lucky few who would be OK to use it.

Anyway, I am open to solutions to solve for everyone, without preventing any systems from getting lower category assignments. As you probably know, Prime95/MPrime solves by this by directly getting another assignment whenever P-1 finds a factor and it runs out of work, as it has the PrimeNet functionality built in.
tdulcet is offline   Reply With Quote
Old 2021-11-09, 03:40   #18
techn1ciaN
 
techn1ciaN's Avatar
 
Oct 2021
U.S. / Maine

89 Posts
Default

Quote:
Originally Posted by tdulcet View Post
Most likely, anything requested will be added to that list, although it would help prioritize what I work on next after the PRP proof uploading.
Work lines without AIDs might be manually loaded into the work file (e.g., someone picking up work from the strategic DC thread). Would it be feasible for your script to register these lines at the next PrimeNet check-in and rewrite them with the AIDs that are returned? Currently, the only way to get AIDs for such lines if you're working them on a GPU is to check them out one by one with the manual assignment tool (tedious; doesn't work for anything that the manual assignment rules prohibit) or to add the lines to your Prime95 work file, let Prime95 reserve and rewrite them, and then move the result (also tedious; can make Prime95 start meaningless P-1 or TF if you use SequentialWorkToDo=2).

Prime95's convention is that adding a work line with no AID will trigger a PrimeNet registration attempt the next time you start the software, unless you write "N/A" where the AID would normally go (to avoid annoying repeat errors with line formats that PrimeNet does not understand yet like Pminus1 / Pplus1.)

I checked your existing to-do list and the only item that seemed like it could be relevant was "Reserve a specific exponent." I don't know whether that refers to this functionality or to something else.
techn1ciaN is online now   Reply With Quote
Old 2021-11-09, 14:22   #19
tdulcet
 
tdulcet's Avatar
 
"Teal Dulcet"
Jun 2018

2·33 Posts
Default

Quote:
Originally Posted by techn1ciaN View Post
Work lines without AIDs might be manually loaded into the work file (e.g., someone picking up work from the strategic DC thread). Would it be feasible for your script to register these lines at the next PrimeNet check-in and rewrite them with the AIDs that are returned? Currently, the only way to get AIDs for such lines if you're working them on a GPU is to check them out one by one with the manual assignment tool (tedious; doesn't work for anything that the manual assignment rules prohibit) or to add the lines to your Prime95 work file, let Prime95 reserve and rewrite them, and then move the result (also tedious; can make Prime95 start meaningless P-1 or TF if you use SequentialWorkToDo=2).
Yes, it would be possible. In my last major update of the PrimeNet script in late August, I implemented about 80% of the code to reserve individual exponents using the register assignment endpoint of the v5 API:
Quote:
Originally Posted by tdulcet View Post
  • Implemented many of the necessary changes to eventually support reserving individual assignments using the register assignment endpoint of the v5 API. Added support for parsing assignments in the worktodo file with N/A, n/a or 0 for the assignment ID and assignments with no ID at all.
I have not finished it, as I am not sure how many people would be interested in manually creating and reserving assignments. I was only interested personally because I wanted our PrimeNet script to support the full functionality of the v5 PrimeNet API. Other than maybe a couple of people on that strategic DC thread, I do not think many people would actually use this functionality. It would also require a refactoring of how the script currently works in order to rewrite the worktodo file and add in the AIDs as you described. This is also why it does not yet support unreserving individual assignments.

Anyway, it does support parsing assignments in the worktodo file without a AID, although it should currently just ignore them. While this is completely unsupported and untested, I believe if you called the register_assignment() function in our script from the correct place, it would register your assignments and output the new AIDs, although you would then need to manually copy/paste those AIDs back into your worktodo file. I am not sure if this would be any easier than those two other methods you listed.

Quote:
Originally Posted by techn1ciaN View Post
Prime95's convention is that adding a work line with no AID will trigger a PrimeNet registration attempt the next time you start the software, unless you write "N/A" where the AID would normally go (to avoid annoying repeat errors with line formats that PrimeNet does not understand yet like Pminus1 / Pplus1.)
Yes, it supports parsing assignments following that convention. However, another reason I decided not to finish implementing this feature is that there is no standard for assignments without an AID that is supported by all of the GIMPS programs. From examining the source code for the programs, here is what is specifically supported:
  • Prime95/MPrime: N/A, N/a, n/A, n/a or no AID.
  • Mlucas: No AID if LL and no P-1 needed.
  • CUDALucas: N/A, N/a, n/A, n/a or no AID.
  • GpuOwl: N/A or 0.
As you can see, there is no convention that is supported by all three of the programs our script supports... BTW, our script does support the Pminus1 and PMinus1 (capital M) assignment formats, as those are used by Mlucas.

Quote:
Originally Posted by techn1ciaN View Post
I checked your existing to-do list and the only item that seemed like it could be relevant was "Reserve a specific exponent." I don't know whether that refers to this functionality or to something else.
Yes, that refers to this functionality. It originality said "Support reserving exponents in a range (#4) or a specific exponent", but I completed the first part of that in my last major update. Anyway, I may finish implementing this feature at some point, but in the meantime, anyone is of course welcome to submit a pull request.

Quote:
Originally Posted by techn1ciaN View Post
I was reluctant to load 6.11 because I used tdulcet's PrimeNet script to register my 5700 XT as a PrimeNet "computer" and have been working towards qualifying it for cat 0 FTCs; I run 7.2 for the fused P-1 since most PRP FTCs seem to come without P-1 done. It occurs to me now, though, that I could simply haul the 6.11 results into the folder with my 7.2 instance and the script, let them be uploaded, and achieve the same result.
To respond to your message in the strategic DC thread, I thought I should note that you could put both versions of GpuOwl in the same folder with our PrimeNet script and then run the correct one for whatever worktype you are currently doing. If you want to keep the GpuOwl versions in separate folders, you could also just copy the script (primenet.py) and then move config file (local.ini) between the folders when switching versions.
tdulcet is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to interface gpuOwl with PrimeNet preda PrimeNet 2 2017-10-07 21:32
primenet account Unregistered Information & Answers 9 2013-04-29 12:32
GIMPS Computer not showing up in account. lewisforlife Software 6 2011-07-24 20:17
computer id missing in Primenet account report onesoul PrimeNet 6 2007-02-13 06:14
GIMPS Teams and Recruiting, Listing Teams eepiccolo Lounge 13 2003-05-02 00:28

All times are UTC. The time now is 16:12.


Tue Nov 30 16:12:45 UTC 2021 up 130 days, 10:41, 0 users, load averages: 1.73, 1.43, 1.44

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.