mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > Conjectures 'R Us

Reply
 
Thread Tools
Old 2017-05-26, 19:47   #188
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

2·52·7·29 Posts
Default

If you stop sr2sieve mid-stream and then resume it from its checkpoint file, it will not remember factors that it has found and so will find additional factors for the same term. That's just a quirk of the program. It doesn't hurt anything.

When hyperthreading, I believe that each core will not know when a different core has found a factor and so you will get more than one factor for the same term on different cores.
gd_barnes is offline   Reply With Quote
Old 2017-05-27, 05:30   #189
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

206078 Posts
Default

Quote:
Originally Posted by gd_barnes View Post
If you stop sr2sieve mid-stream and then resume it from its checkpoint file, it will not remember factors that it has found and so will find additional factors for the same term. That's just a quirk of the program. It doesn't hurt anything.

When hyperthreading, I believe that each core will not know when a different core has found a factor and so you will get more than one factor for the same term on different cores.
The first part of your post may be the real reason, I may have been restarted the sr2sieve a couple of times during the tests and forgot to clear the checkpoint file. If that, than sorry for the false ararm. , and thanks for pointing it out.

However the second part of your post doesn't make sense. The instances of the process were separate (different bases, remember, so there were 21 tasks running, no shared memory, as the memory is process-bonded not core-bonded. If changing the core would cause the program losing data, for example every time when the afiinity changes, then that would be a serious bug, not only in the program, but in the whole system itself (computer, hardware, software, all the concept).
LaurV is offline   Reply With Quote
Old 2017-05-27, 07:12   #190
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

2·52·7·29 Posts
Default

You can ignore the second paragraph of my post. You're right. It makes no sense. I was thinking multi-core on one base...not hyperthreading one base on a single (or multiple) cores. If you're running multiple cores on one base, one core would not know of the factors found by the other cores. Anyway, oops. :-)

After reading through the thread I am near 100% sure that my first paragraph is what happened to you. But the remedy that you mentioned would not work when stopping sr2sieve mid-stream. It doesn't matter what you do to the checkpoint file, sr2sieve will not "remember" previous factors found when you restart it. While it is running it stores them in memory so you will get no duplicates if you let it run through to the end. When it finishes or you stop it mid-stream it clears out memory although the factors.txt file is still there. When restarted, the factors.txt file will then be appended to with potentially more than one factor for a term becaue sr2sieve will not know which factors have been previously found before it was stopped.

Last fiddled with by gd_barnes on 2017-05-27 at 07:15
gd_barnes is offline   Reply With Quote
Old 2017-05-27, 08:55   #191
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

3·2,861 Posts
Default

Quote:
Originally Posted by gd_barnes View Post
But the remedy that you mentioned would not work when stopping sr2sieve mid-stream. It doesn't matter what you do to the checkpoint file, sr2sieve will not "remember" previous factors found when you restart it.
yes, but when you delete the ckpoint, it will start from scratch (as the input file is not changed unless I manually elliminate the factors with srfile). That is easy to see in the factor list: all factors repeat themselves from the beginning, not just one single lost factor somewhere (that happened to me too, and I manually edited the factor file to eliminate the duplicate bunch at the end, that is a different thing). We are good now. Sorry again for the false alarm.

Last fiddled with by LaurV on 2017-05-27 at 08:58 Reason: fix quote tag
LaurV is offline   Reply With Quote
Old 2017-05-30, 15:50   #192
sweety439
 
sweety439's Avatar
 
Nov 2016

23×89 Posts
Default

Quote:
Originally Posted by LaurV View Post
Because we anyhow totally hijacked this thread...
(I elliminated the macros and DDE part, you can click on the plus signs or on the little 1/2 tabs on the upper left, or see the second sheet too)
@LaurV:

Can you update a file like this? For all bases 2<=b<=2048.

Besides, what is your search limit? 100K? 200K? 500K? or 1M?
Attached Files
File Type: txt least k such that 2n^k-1 is prime.txt (2.9 KB, 37 views)

Last fiddled with by sweety439 on 2017-05-30 at 16:12
sweety439 is offline   Reply With Quote
Old 2017-05-31, 02:18   #193
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

218716 Posts
Default

Quote:
Originally Posted by sweety439 View Post
@LaurV:

Can you update a file like this? For all bases 2<=b<=2048.

Besides, what is your search limit? 100K? 200K? 500K? or 1M?
Did you open the file I posted? Did you do any of the actions I said in the post that you can do? (i.e. clicking on the little 1/2 icons in the upper left corners, clicking on the +/- signs, etc?) Or did I just wasted my time to "clean" it from the macros and DDE stuff that brings the results into it, to make it "safe" for you to open? (no, I don't put the results manually into it). That file was posted mostly for you (and partially for Gary). Look inside. What you need is just a mark all, copy, paste.

Last fiddled with by LaurV on 2017-05-31 at 02:19
LaurV is offline   Reply With Quote
Old 2017-05-31, 15:31   #194
KEP
Quasi Admin Thing
 
KEP's Avatar
 
May 2005

3·5·61 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
Yay! Glad to see another one of those fall - it's been a while!



Yes, it has been a while. Kind of makes me hope that there could hide 1 more prime in the bulk of tests, for the remaining 9 k's I'm currently testing on the Riesel side to n=750K.

On a sidenote, the LLR 3.8.20 going live thread at primegrid, offers a solution to run LLR multithreaded on PRPnet. It could increase the testing throughput by a lot. I did by switching from 3 cores running a single thread to running 2 instances of 2 threads, increase my overall production by 70%/day and by converting my base 16 numbers to plain base 2 numbers, before starting up LLR, I did increase my productivity by an additional 25%/day (more or less), compared to testing the same numbers as base 16 numbers.

A bit off-topic, but nice to know for those who does want to increase their overall productivity for a lesser heat production also

Thanks for your gratulation

Take care.
KEP is offline   Reply With Quote
Old 2017-05-31, 18:59   #195
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

11000011010012 Posts
Default

Quote:
Originally Posted by KEP View Post
On a sidenote, the LLR 3.8.20 going live thread at primegrid, offers a solution to run LLR multithreaded on PRPnet. It could increase the testing throughput by a lot. I did by switching from 3 cores running a single thread to running 2 instances of 2 threads, increase my overall production by 70%/day and by converting my base 16 numbers to plain base 2 numbers, before starting up LLR, I did increase my productivity by an additional 25%/day (more or less), compared to testing the same numbers as base 16 numbers.

A bit off-topic, but nice to know for those who does want to increase their overall productivity for a lesser heat production also
Is there a good "rule of thumb" for determining when it's better to use fewer instances with more threads, versus more instances with one thread each?

My "simple" understanding of this is that it's better to run one LLR on each core with one thread each, because there are plenty of separate candidates to keep all cores busy, and that way you have no losses due to the imperfect scaling of parallelizing a single test across multiple cores. Based on this, I always understood multithreaded LLR to be something primarily useful for when we get to "really huge" tests (i.e., GIMPS or SoB level) when it becomes more important to (e.g.) verify a single prime in the shortest amount of time than to maximize overall throughput.

But perhaps this understanding is outdated. Is it a matter of memory bandwidth that makes multithreading useful even for our (relatively) "small" tests? Or is it something to do with hyperthreading?
mdettweiler is offline   Reply With Quote
Old 2017-05-31, 19:10   #196
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

22×3×107 Posts
Default

If you have "normal" Intel CPU with 6 or 8 MB L3 cache size, then start using t-2 swich only above 256-288K. T-2 can hold up to 480K or even 512K and then use T-3 up to 768K
So it is basically 256K per core or per 1.5MB of cache size.
But that is only me :)
When you take for example 320 K candidate per core on quad core CPU it goes to overhead, and it become to slow down. Prime95 support T-2 switch for long of time, and I use it. Now LLR can do the same.

Last fiddled with by pepi37 on 2017-05-31 at 19:13
pepi37 is offline   Reply With Quote
Old 2017-06-01, 18:07   #197
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3·2,083 Posts
Default

OK, so if I'm understanding you correctly - the goal is to keep the working set of all LLRs (across all cores) within the L3 cache size of the CPU, right? And this scales (roughly) with the FFT size?

So...one of my computers is a dual-core Sandy Bridge with 4 MB of L3 cache. It is testing Riesel base 23 around n=1.4M, which is using a 560K FFT size. 560K * 2 = 1120K, which is well within the 4 MB of L3 cache. Does that mean I should keep using 2 separate threads instead of -t2?

Obviously the answer would be different for a quad-core, since it has has not much more cache and more cores to share it with.

Actually...should I be running *4* separate clients on this machine? 560K * 4 = 2240K, which still fits within L3 cache. The machine has 2 physical cores but 4 logical cores due to hyperthreading. I had always understood that LLR didn't gain much from hyperthreading, but since LLR performance is memory-bandwidth-intensive, perhaps there is some room to gain there...maybe I should try this.


(P.S.: Gary, you might want to move this exchange to the "Software/Instructions/Questions" thread, since it's a bit off-topic here...)

Last fiddled with by mdettweiler on 2017-06-01 at 18:10
mdettweiler is offline   Reply With Quote
Old 2017-06-01, 19:02   #198
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

22·3·107 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
So...one of my computers is a dual-core Sandy Bridge with 4 MB of L3 cache. It is testing Riesel base 23 around n=1.4M, which is using a 560K FFT size. 560K * 2 = 1120K, which is well within the 4 MB of L3 cache. Does that mean I should keep using 2 separate threads instead of -t2?
So your candidate has 560K and you are on dual core machine, so one core has 2 MB of L3 cache .
Lets do math: 288K is 1 MB, 576K is 2MB and since you have 4MB and two cores, answer is: you should use -t2


Quote:
Originally Posted by mdettweiler View Post
Actually...should I be running *4* separate clients on this machine? 560K * 4 = 2240K, which still fits within L3 cache. The machine has 2 physical cores but 4 logical cores due to hyperthreading. I had always understood that LLR didn't gain much from hyperthreading, but since LLR performance is memory-bandwidth-intensive, perhaps there is some room to gain there...maybe I should try this.
For me , and for many on this forum there is no HT, this is fiction. And fiction and reality doesnot match at all.
If you use 4 candidates in parallel then
288 K =1 M, 576 K = 2M
So you should need to have 8 MB of cache and you only have 4 MB of L3 cache
pepi37 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Useless SSE instructions __HRB__ Programming 41 2012-07-07 17:43
Questions about software licenses... WraithX GMP-ECM 37 2011-10-28 01:04
Software/instructions/questions gd_barnes No Prime Left Behind 48 2009-07-31 01:44
Instructions to manual LLR? OmbooHankvald PSearch 3 2005-08-05 20:28
Instructions please? jasong Sierpinski/Riesel Base 5 10 2005-03-14 04:03

All times are UTC. The time now is 02:38.

Wed Jul 8 02:38:44 UTC 2020 up 105 days, 11 mins, 0 users, load averages: 1.90, 2.29, 2.33

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.