mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > Conjectures 'R Us

Reply
 
Thread Tools
Old 2022-12-09, 01:41   #23
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

11,087 Posts
Default

Quote:
Originally Posted by storm5510 View Post
So, I begin, again. Live and learn...
It happens. From time to time. It is how we learn... 8^)
chalsall is offline   Reply With Quote
Old 2022-12-09, 02:15   #24
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009
Not U. + S.A.

2×7×181 Posts
Default

So, here is what I have set up:

Base 60

Machine 1 runs -p 3 to -P 4e11
Machine 2 runs -p 4e11 to -P 650e9
Machine 3 runs -p 650e9 to -P 135e10

They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase.

Each will produce a .abcd file with slightly different names. I can then merge them together into a single file.

Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me.

I hope this is what everyone has in mind.

Edit: The below is what I referred to as "stacking:"

Code:
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "36*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "1700*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "4708*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5317*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5611*60^n-1"

Last fiddled with by storm5510 on 2022-12-09 at 02:36 Reason: Additional
storm5510 is offline   Reply With Quote
Old 2022-12-09, 02:54   #25
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

2·34·43 Posts
Default

Quote:
Originally Posted by storm5510 View Post
So, here is what I have set up:

Base 60

Machine 1 runs -p 3 to -P 4e11
Machine 2 runs -p 4e11 to -P 650e9
Machine 3 runs -p 650e9 to -P 135e10

They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase.

Each will produce a .abcd file with slightly different names. I can then merge them together into a single file.

Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me.

I hope this is what everyone has in mind.
Yes, something like this:

srsieve2cl -n 100e3 -N 250e3 -P 135e10 -g 16 -M 9000 -s "36*60^n-1" -s "1700*60^n-1" -s"4708*60^n-1" -s "5317*60^n-1" -s "5611*60^n-1"

or

srsieve2cl -n 100e3 -N 250e3 -P 135e10 -g 16 -M 9000 -s b60.in

where b60.in is a file where each line is a separate sequence for the same base.

If you are going to sieve the same base across multiple computers, pre-sieve to 1e9 to eliminate 90% of the factors. Use that output file as input to each machine so you get

srsieve2 -p1e9 -P4e11 -i b60_n.abcd -O f1.txt
srsieve2 -p4e11 -P8e11 -i b60_n.abcd -O f2.txt
srsieve2 -p8e11 -P12e11 -i b60_n.abcd -O f3.txt

When all of those are done:

srsieve2 -A -i b60_n.abcd -I f1.txt
srsieve2 -A -i b60_n.abcd -I f2.txt
srsieve2 -A -i b60_n.abcd -I f3.txt

srsieve2 and srsieve2cl are interchangeable, but if the machines are different, speed wise, then the range of p will differ.

My recommendation is to run different bases on different machines so that you don't need to piece together factor files. I think you will be surprised at how quickly you can sieve to 1e12 or 1e13 when you have dozens or hundreds of sequences on a GPU. I could sieve over 2000 sequences to 1e12 in less than 3 days on a GPU.
rogue is offline   Reply With Quote
Old 2022-12-09, 04:58   #26
gd_barnes
 
gd_barnes's Avatar
 
"Gary"
May 2007
Overland Park, KS

3×11×359 Posts
Default

Quote:
Originally Posted by storm5510 View Post
So, here is what I have set up:

Base 60

Machine 1 runs -p 3 to -P 4e11
Machine 2 runs -p 4e11 to -P 650e9
Machine 3 runs -p 650e9 to -P 135e10

They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase.

Each will produce a .abcd file with slightly different names. I can then merge them together into a single file.

Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me.

I hope this is what everyone has in mind.

Edit: The below is what I referred to as "stacking:"

Code:
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "36*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "1700*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "4708*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5317*60^n-1"
srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5611*60^n-1"
No. It is not what we have in mind. Without using the -O command, the first part of it will not even work properly. You are not creating factor files but instead sieve files. How will you be able to "merge" together sieve files for the same base that have different tests remaining? Merging them won't logically work. You have to create factor files and merge those together and then use those to remove terms from the sieve file.

Why do you insist on running one k at a time as you show in the last part of your process? We don't want to continue helping you if you insist on running one k at a time no matter how many batch jobs you set up. Please trust us that the below is the way to do this.

I will create an input file for you and spell it out as best as I can.

1. Using the attached file of k's called R60-remain.txt, run the following command on machine 1:
srsieve2cl -n 100e3 -N 250e3 -P 1e9 -g 16 -M 9000 -s R60-remain.txt

This will run quickly and is a precursor to running the big jobs next. It will create an output file. I suggest renaming it to something relevant. Let's call it R60-sieve.txt

The following assumes that you have 3 equal machines. You can tweak the sieving ranges to make them complete at about the same time.

2. Using the examples that Mark gave, run the following command on machine 1:
srsieve2cl -p1e9 -P4e11 -i R60-sieve.txt -O factors1.txt
(tweak as needed for the relevant number of cores/threads)

3. Run the following command on machine 2:
srsieve2cl -p4e11 -P8e11 -i R60-sieve.txt -O factors2.txt

4. Run the following command on machine 3:
srsieve2cl -p8e11 -P12e11 -i R60-sieve.txt -O factors3.txt

5. You now have 3 factor files factors1.txt, factors2.txt, and factors3.txt. Merge them all together. I just copy and paste each one to the end of the first one. To check yourself, add up how many total lines are in all of the files. Then after the copy and paste, make sure your final file has that many lines. You can also use a DOS command (or there's probably a myriad of other ways) to merge all of the files together. Rename that merged file as factors-all.txt

6. Run the following command:
srsieve2cl -A -i R60-sieve-txt -I factors-all.txt

This will create a new output file with all of the factors removed.

7. Rename the output file from #6 to something relevant again like R60-sieveb.txt

8. Correct the sieve depth in R60-sieveb.txt to show 120000000000 since you sieved to 12e11.

9. Run an LLR test at 60% of the n-range using approximately the median k-value in the file (60% of n=100000 to 250000 = 190000 and the median k is 12061, so use 12061*60^190000-1 in this case) to see how long a test takes.

10. Begin a short trial sieve using R60-sieveb.txt to see what the removal rate is. If you are removing factors at twice the rate of an LLR test, then you need to sieve to slightly less than double the current sieve depth. Note that this is not exact math-wise but it gets you close. If you are way off, say you are removing factors 10 times faster than the LLR test, don't automatically sieve 10 times as far. Do something less than that, say 5-7 times as far, and then see what the removal rate is again. Nothing is ever quite a straight-line when it comes to a removal rate.

11. Delete all extraneous factor or sieve files except for R60-sieveb.txt

12. Repeat steps 2 thru 11 as many times as necessary until you've reached approximately the correct optimum sieve depth. On the 2nd time through, use the new file R60-sieveb.txt. Each time you go through it, continue using the final file from step 8 as input to the new step 2.

Have you looked at some of the documentation that comes with the sieving programs, such as README? If not, that can be very helpful in understanding the difference between factor files and sieve files and how to merge and remove one from the other.
Attached Files
File Type: txt R60-remain.txt (446 Bytes, 11 views)

Last fiddled with by gd_barnes on 2022-12-09 at 05:13
gd_barnes is online now   Reply With Quote
Old 2022-12-09, 05:15   #27
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009
Not U. + S.A.

2×7×181 Posts
Default

@gd_barnes

OK, all of this is sailing over my head so it would probably be best to just drop the assignment. Please!
storm5510 is offline   Reply With Quote
Old 2022-12-09, 05:32   #28
gd_barnes
 
gd_barnes's Avatar
 
"Gary"
May 2007
Overland Park, KS

3×11×359 Posts
Default

Quote:
Originally Posted by storm5510 View Post
@gd_barnes

OK, all of this is sailing over my head so it would probably be best to just drop the assignment. Please!
OK well...sorry to hear that.

What part of it is confusing you? Have you done factor removal from sieve files before? Once you've done it a couple of times, it becomes a lot easier.

On the optimum sieve depth part, if you want to skip that and just provide us with a preliminary sieve file, say, sieved to 1e12 or 5e12 or 10e12, then that would be OK too. Others can then pick up the sieve file, determine optimum sieve depth, and finish it off.
gd_barnes is online now   Reply With Quote
Old 2022-12-09, 05:51   #29
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009
Not U. + S.A.

2·7·181 Posts
Default

Quote:
Originally Posted by gd_barnes View Post
OK well...sorry to hear that.

What part of it is confusing you? Have you done factor removal from sieve files before? Once you've done it a couple of times, it becomes a lot easier.

On the optimum sieve depth part, if you want to skip that and just provide us with a preliminary sieve file, say, sieved to 1e12 or 5e12 or 10e12, then that would be OK too. Others can then pick up the sieve file, determine optimum sieve depth, and finish it off.
Scratch my request! It was a decision made in haste because of being really tired.

What I need to do is study your last instructional post in detail, follow it, and see what I can come up with.

I had an issue with srsieve2cl which I have submitted for rogue to look into. I found a condition where it refused to start and displayed an error message.

Now, sleep!
storm5510 is offline   Reply With Quote
Old 2022-12-09, 07:48   #30
gd_barnes
 
gd_barnes's Avatar
 
"Gary"
May 2007
Overland Park, KS

271078 Posts
Default

The instructions that I gave are essentially a turn-key operation of a complete sieving process using multiple machines. Doing all of that, the file would be completely ready for LLR testing in the most efficient way possible, both from a personal time and CPU time perspective.

Mark had suggested that you might find it easier to run one base on only one machine. That is true. But I had the impression that you wanted to use multiple machines for a single base.

If you want to skip the optimum sieve calculations, that is OK. I can guarantee that the optimum sieve depth is > 10e12, likely > 20e12. srsieve2cl is amazingly fast for multiple k's on multiple cores so it could be higher than that but I cannot say for sure without setting it up myself.

To make it worth your while, you should at least sieve to 1e12. If you can sieve to 10e12, that would be great. That's the maximum I'd suggest that you do without computing an optimum sieve depth.

There is an alternative that you might find easier that is only a little less efficient CPU-wise. Split that k-file up that I posted (R60-remain.txt) into 3 files, 1 for each machine, of 11 k's each or whatever ratio works since you have machines of varying capabilities. You still get the efficiency of multiple k's in each sieve without having to copy factor and sieve files across multiple machines, albeit somewhat less efficient.

Doing this, you'd have 3 separate sieves. You'd have separate factor files to remove from the separate sieve files each time they finish. That might make it a little easier to wrap your head around.

Then when all 3 sieves are done sieving to the same depth, you can use srfile to combine them into one big sieve file to send to us.
gd_barnes is online now   Reply With Quote
Old 2022-12-09, 16:55   #31
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009
Not U. + S.A.

1001111001102 Posts
Default

I was not able to sleep last night so I got back up to work on this. I was up until 5 AM EST.

I took a lot from the post rogue made. I created a list as he mentioned and called it "input.txt"

Code:
36*60^n-1
1700*60^n-1
4708*60^n-1
5317*60^n-1
5611*60^n-1
6101*60^n-1
6162*60^n-1
6274*60^n-1
7060*60^n-1
7870*60^n-1
I developed my batch process as I read along, one step at a time. and testing each line individually. Below is the batch from Machine 3: 1 and 2 as different as they are not running a GPU version. The basics are the same.

Code:
@echo off
cls
srsieve2cl -n 100e3 -N 250e3 -p 1e12 -P 2e12 -g 16 -M 9000 -s input.txt
srsieve2cl -p 1e12 -P 2e12 -g 16 -M 9000 -i b60_n.abcd -O f3.txt
srsieve2cl -A -g 16 -M 9000 -i b60_n.abcd -I f3.txt
I wrote these in such a way to make them reusable by simply changing the input file. Below are the "p" settings for each.

Code:
Machine 1: -p 3 -P 45e10.
Machine 2: -p 45e10 to -P 1e12. 
Machine 3: -p 1e12 to 2e12.
All three should finish in a 3-hour window tomorrow.
storm5510 is offline   Reply With Quote
Old 2022-12-09, 17:40   #32
rogue
 
rogue's Avatar
 
"Mark"
Apr 2003
Between here and the

11011001101102 Posts
Default

Once you do the initial sieving to 1e9, then the first instance will start with 1e9. All three instances will the output from the the initial sieving as their input.
rogue is offline   Reply With Quote
Old 2022-12-09, 18:10   #33
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009
Not U. + S.A.

253410 Posts
Default

Quote:
Originally Posted by rogue View Post
Once you do the initial sieving to 1e9, then the first instance will start with 1e9. All three instances will the output from the the initial sieving as their input.
You lose me here. I am not doing any sieving which stops at 1e9. My first batch is from 3 to 45e10.
storm5510 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sieving for CRUS rebirther Conjectures 'R Us 828 2023-02-05 16:56
Two sieving questions The Carnivore Twin Prime Search 13 2019-07-06 03:11
Sieving Drive all base 2/4 k's worked by CRUS gd_barnes Conjectures 'R Us 143 2014-10-21 23:55
Test Sieving Questions nstaab1 Lounge 15 2013-03-06 13:48
CRUS-like sieving challenge CRGreathouse Puzzles 24 2011-10-28 18:30

All times are UTC. The time now is 06:20.


Mon Feb 6 06:20:08 UTC 2023 up 172 days, 3:48, 1 user, load averages: 1.76, 1.27, 1.13

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎𝜍 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔