Software/instructions/questions

[gd_barnes]

If you stop sr2sieve mid-stream and then resume it from its checkpoint file, it will not remember factors that it has found and so will find additional factors for the same term. That's just a quirk of the program. It doesn't hurt anything.

When hyperthreading, I believe that each core will not know when a different core has found a factor and so you will get more than one factor for the same term on different cores.

[LaurV]

Originally Posted by gd_barnes

If you stop sr2sieve mid-stream and then resume it from its checkpoint file, it will not remember factors that it has found and so will find additional factors for the same term. That's just a quirk of the program. It doesn't hurt anything.

When hyperthreading, I believe that each core will not know when a different core has found a factor and so you will get more than one factor for the same term on different cores.

The first part of your post may be the real reason, I may have been restarted the sr2sieve a couple of times during the tests and forgot to clear the checkpoint file. If that, than sorry for the false ararm. , and thanks for pointing it out.

However the second part of your post doesn't make sense. The instances of the process were separate (different bases, remember, so there were 21 tasks running, no shared memory, as the memory is process-bonded not core-bonded. If changing the core would cause the program losing data, for example every time when the afiinity changes, then that would be a serious bug, not only in the program, but in the whole system itself (computer, hardware, software, all the concept).

[gd_barnes]

You can ignore the second paragraph of my post. You're right. It makes no sense. I was thinking multi-core on one base...not hyperthreading one base on a single (or multiple) cores. If you're running multiple cores on one base, one core would not know of the factors found by the other cores. Anyway, oops. :-)

After reading through the thread I am near 100% sure that my first paragraph is what happened to you. But the remedy that you mentioned would not work when stopping sr2sieve mid-stream. It doesn't matter what you do to the checkpoint file, sr2sieve will not "remember" previous factors found when you restart it. While it is running it stores them in memory so you will get no duplicates if you let it run through to the end. When it finishes or you stop it mid-stream it clears out memory although the factors.txt file is still there. When restarted, the factors.txt file will then be appended to with potentially more than one factor for a term becaue sr2sieve will not know which factors have been previously found before it was stopped.

[LaurV]

Originally Posted by gd_barnes

But the remedy that you mentioned would not work when stopping sr2sieve mid-stream. It doesn't matter what you do to the checkpoint file, sr2sieve will not "remember" previous factors found when you restart it.

yes, but when you delete the ckpoint, it will start from scratch (as the input file is not changed unless I manually elliminate the factors with srfile). That is easy to see in the factor list: all factors repeat themselves from the beginning, not just one single lost factor somewhere (that happened to me too, and I manually edited the factor file to eliminate the duplicate bunch at the end, that is a different thing). We are good now. Sorry again for the false alarm.

[sweety439]

Originally Posted by LaurV

Because we anyhow totally hijacked this thread...
(I elliminated the macros and DDE part, you can click on the plus signs or on the little 1/2 tabs on the upper left, or see the second sheet too)

@LaurV:

Can you update a file like this? For all bases 2<=b<=2048.

Besides, what is your search limit? 100K? 200K? 500K? or 1M?

[LaurV]

Originally Posted by sweety439

@LaurV:

Can you update a file like this? For all bases 2<=b<=2048.

Besides, what is your search limit? 100K? 200K? 500K? or 1M?

Did you open the file I posted? Did you do any of the actions I said in the post that you can do? (i.e. clicking on the little 1/2 icons in the upper left corners, clicking on the +/- signs, etc?) Or did I just wasted my time to "clean" it from the macros and DDE stuff that brings the results into it, to make it "safe" for you to open? (no, I don't put the results manually into it). That file was posted mostly for you (and partially for Gary). Look inside. What you need is just a mark all, copy, paste.

[KEP]

Originally Posted by mdettweiler

Yay! Glad to see another one of those fall - it's been a while!

Yes, it has been a while. Kind of makes me hope that there could hide 1 more prime in the bulk of tests, for the remaining 9 k's I'm currently testing on the Riesel side to n=750K.

On a sidenote, the LLR 3.8.20 going live thread at primegrid, offers a solution to run LLR multithreaded on PRPnet. It could increase the testing throughput by a lot. I did by switching from 3 cores running a single thread to running 2 instances of 2 threads, increase my overall production by 70%/day and by converting my base 16 numbers to plain base 2 numbers, before starting up LLR, I did increase my productivity by an additional 25%/day (more or less), compared to testing the same numbers as base 16 numbers.

A bit off-topic, but nice to know for those who does want to increase their overall productivity for a lesser heat production also

Thanks for your gratulation

Take care.

[mdettweiler]

Originally Posted by KEP

On a sidenote, the LLR 3.8.20 going live thread at primegrid, offers a solution to run LLR multithreaded on PRPnet. It could increase the testing throughput by a lot. I did by switching from 3 cores running a single thread to running 2 instances of 2 threads, increase my overall production by 70%/day and by converting my base 16 numbers to plain base 2 numbers, before starting up LLR, I did increase my productivity by an additional 25%/day (more or less), compared to testing the same numbers as base 16 numbers.

A bit off-topic, but nice to know for those who does want to increase their overall productivity for a lesser heat production also

Is there a good "rule of thumb" for determining when it's better to use fewer instances with more threads, versus more instances with one thread each?

My "simple" understanding of this is that it's better to run one LLR on each core with one thread each, because there are plenty of separate candidates to keep all cores busy, and that way you have no losses due to the imperfect scaling of parallelizing a single test across multiple cores. Based on this, I always understood multithreaded LLR to be something primarily useful for when we get to "really huge" tests (i.e., GIMPS or SoB level) when it becomes more important to (e.g.) verify a single prime in the shortest amount of time than to maximize overall throughput.

But perhaps this understanding is outdated. Is it a matter of memory bandwidth that makes multithreading useful even for our (relatively) "small" tests? Or is it something to do with hyperthreading?

[pepi37]

If you have "normal" Intel CPU with 6 or 8 MB L3 cache size, then start using t-2 swich only above 256-288K. T-2 can hold up to 480K or even 512K and then use T-3 up to 768K
So it is basically 256K per core or per 1.5MB of cache size.
But that is only me :)
When you take for example 320 K candidate per core on quad core CPU it goes to overhead, and it become to slow down. Prime95 support T-2 switch for long of time, and I use it. Now LLR can do the same.

[mdettweiler]

OK, so if I'm understanding you correctly - the goal is to keep the working set of all LLRs (across all cores) within the L3 cache size of the CPU, right? And this scales (roughly) with the FFT size?

So...one of my computers is a dual-core Sandy Bridge with 4 MB of L3 cache. It is testing Riesel base 23 around n=1.4M, which is using a 560K FFT size. 560K * 2 = 1120K, which is well within the 4 MB of L3 cache. Does that mean I should keep using 2 separate threads instead of -t2?

Obviously the answer would be different for a quad-core, since it has has not much more cache and more cores to share it with.

Actually...should I be running *4* separate clients on this machine? 560K * 4 = 2240K, which still fits within L3 cache. The machine has 2 physical cores but 4 logical cores due to hyperthreading. I had always understood that LLR didn't gain much from hyperthreading, but since LLR performance is memory-bandwidth-intensive, perhaps there is some room to gain there...maybe I should try this.


(P.S.: Gary, you might want to move this exchange to the "Software/Instructions/Questions" thread, since it's a bit off-topic here...)

[pepi37]

Originally Posted by mdettweiler

So...one of my computers is a dual-core Sandy Bridge with 4 MB of L3 cache. It is testing Riesel base 23 around n=1.4M, which is using a 560K FFT size. 560K * 2 = 1120K, which is well within the 4 MB of L3 cache. Does that mean I should keep using 2 separate threads instead of -t2?

So your candidate has 560K and you are on dual core machine, so one core has 2 MB of L3 cache .
Lets do math: 288K is 1 MB, 576K is 2MB and since you have 4MB and two cores, answer is: you should use -t2

Originally Posted by mdettweiler

Actually...should I be running *4* separate clients on this machine? 560K * 4 = 2240K, which still fits within L3 cache. The machine has 2 physical cores but 4 logical cores due to hyperthreading. I had always understood that LLR didn't gain much from hyperthreading, but since LLR performance is memory-bandwidth-intensive, perhaps there is some room to gain there...maybe I should try this.

For me , and for many on this forum there is no HT, this is fiction. And fiction and reality doesnot match at all.
If you use 4 candidates in parallel then
288 K =1 M, 576 K = 2M
So you should need to have 8 MB of cache and you only have 4 MB of L3 cache