Software suggestion:
 

I assume it's easy for prime95 to be able to check what percent of the computer's proc processing power is being used. I don't know about all of you, but my P-1 factoring doesn't use all of my processing power, so I set it to do trial factoring at the same time.

My suggestion is to make this a default: If Prime95 detects that some of the processing power isn't being used, then simultaneously do other work, like simultaneously trial factor the same number it's doing P-1 work on if there is trial factoring left to do, or do work on another number if there isn't.

[quote=dominicanpapi82;161737]I assume it's easy for prime95 to be able to check what percent of the computer's proc processing power is being used.[/quote]No, it takes the operating system to do that. Maybe there's a call an application can make to get that info from the OS, but aside from that an application can't know about all the stuff going on outside itself.

Now, there [U]is[/U] something an application can do to [I]guess[/I] that the system might be heavily loaded, but it only applies to applications with very predictable workloads like Prime95: periodically check the elapsed time needed for its iterations. When that time significantly increases from average, it's a good guess that there's a load on the system from something other than the application. But that's just an inference, not direct knowledge. (Prime95 already does that under certain circumstances.)

[quote]I don't know about all of you, but my P-1 factoring doesn't use all of my processing power, so I set it to do trial factoring at the same time.[/quote]... on a different worker thread, right?

That doesn't require knowledge of the system load or even have any relationship to it; it's just a setup option that already exists. In fact, Prime95/PrimeNet will do that automatically if the work type settings allow it (specify a default work type of TF for one thread, default work type of P-1 for another, ...).

[quote]like simultaneously trial factor the same number it's doing P-1 work on if there is trial factoring left to do[/quote]There's a reason for doing the sequence

TF-except-last-2-bit-levels,

then P-1,

then TF-last-2-bit-levels.

GIMPS throughput is slightly more efficient that way than if all TF bit levels were done before the P-1.

IIRC, PrimeNet won't even assign both TF and P-1 (or any two different work types) at the same time on the same exponent.

Furthermore, there is no convenience or savings anyway in doing TF on the same number that's being P-1'ed -- the algorithms, arithmetic, code paths and register utilizations are [U]very[/U] different for those two methods, and it's better (for the sequencing reason) to do TF on some different number than what is being P-1'ed. Even when hyperthreading might give you a few percent gain from partial overlap of code execution on the same CPU, there's still no advantage to doing TF and P-1 on the same exponent instead of on different exponents (and there's still the sequencing disadvantage).

I think you misunderstood. I'm not saying there is a big load on my computer taking up processor power from prime95. I'm saying the system idol process has a number around 45% when doing stage 2 of P-1 testing.

[QUOTE=cheesehead;161769]
... on a different worker thread, right? [/quote]

Right.

[QUOTE=cheesehead;161769]
That doesn't require knowledge of the system load or even have any relationship to it; it's just a setup option that folks are already doing, as described elsewhere (check the "Prime95 Version 25.7", 25.8" threads).
[/quote]

I mean as a default option, dependent on how much processing power prime95 is taking vs how much the system idol process is taking.

[QUOTE=cheesehead;161769]
...there is no convenience or savings anyway in doing TF on the same number that's being P-1'ed...
[/quote]
There is if 50% of the processor cycles are not being utilized.

[quote=dominicanpapi82;161773]I think you misunderstood.[/quote]I re-edited my post several times. Read it again as it now stands.

[quote=dominicanpapi82;161773]I'm saying the system idol process has a number around 45% when doing stage 2 of P-1 testing.[/quote]So, if it's a dual-core system, maybe one is fully loaded (50% of system total) with the P-1, and the other is doing only 10% of its potential (5% of system total) on something else, with the system idle process soaking up the remaining 90% of the second core (45% of total).

[quote]I mean as a default option, dependent on how much processing power prime95 is taking vs how much the system idol process is taking.[/quote]Hunh? The system idle process simply uses whatever is not being used by any application on a given core. To fully use two cores, simply assign Prime95 work on both of them ... which I thought you're already doing ("my P-1 factoring doesn't use all of my processing power, so I set it to do trial factoring at the same time", "... on a different worker thread, right?", "Right"). If you have P-1 and TF assigned to two threads _on the same core_, well -- that's a mistake -- put them on different cores.

- - -

Please post your local.txt, prime.txt and worktodo.txt files (after masking out info like passwords, IDs, ComputerGUID, assignment hex strings, & specific exponent numbers) so we can see your setup.

This is what I mean (see the attachment) :smile::

[quote=dominicanpapi82;161780]This is what I mean (see the attachment) :smile::[/quote]If it's a dual-core system, then it looks like one is practically fully loaded (47% of system total = 94% of one core) with Prime95. If it's a quad-core, then Prime95 is running threads on only two of them (e.g., 94% of one core plus 94% of another core plus 0% of the other two cores = 47% of total), so to get more Prime95 work done, you have to define two more worker threads and give each of them an assignment.

The problem occurs when you have one worker using two threads and that worker is busy with a single threaded task (P-1 or TF).

One can have three types of work assigne for one exponent : an exponent assigned for LL testing that has not beeen TF'ed to optimal limits and for wich no P-1 work has been done. There are at least two implied extra assignmens then : TF and P-1. From what I remember in such a case it does the TF in one run, not stopping for P-1 and finishing TF after an unsuccessful P-1.

Jacob

One can also get low CPU utilization if your configuration (memory allocation) causes excessive page file use; the CPU is idle while it waits for data from the HDD.

I think you all are looking for a solution to a problem without understanding what I'm saying.

I'm not saying it's a hardware or windows issue. It's not a page file issue, my processing power isn't being taken up by another program, and (if it comes up) I'm not throttling my system. It's not anything like that. Prime95 is running as fast as it's able to, and it's still leaving unused cycles.

Prime95 gives you the option to work on just one number because, well, you might want to work on just one number. I want to do work on just one number. Not that I need justification because many other people also want to work on just one number, but I want to work on just one number because I am working on a 100M digit mersenne that takes years to finish.

If it's true that P-1 testing can't take full advantage of multiple cores, then given that some multicore computers will be set up to work on just one number, it will definitely be an improvement in throughput (by weeks for 100M digit mersennes) if we are able to simultaneously P-1 test and trial factor the same number. I don't mean with separate code, or some slick engineering to try to save steps if that is at all possible. I mean, just do both on the same number simultaneously just like you could do P-1 testing on one mersenne and trial factoring on another simultaneously. The average PC would save time processing a single number if it simultaneously does P-1 factoring and trial factoring.

[quote=dominicanpapi82;161871]I think you all are looking for a solution to a problem without understanding what I'm saying.[/quote]But you haven't yet said anything that shows us we are wrong. We see no reason to think that you aren't simply overlooking a hardware, Windows, or setup cause of your complaints. Until you show us evidence that rules out a hardware, Windows, or setup cause, we're going to continue to suspect that one of those is the culprit rather than the more exotic and less-believable reason you're proposing.

(After all, you led off this thread by stating that you assumed something which isn't true, so from our points of view it seems reasonable to think that might not be the only mistaken assumption you're making. We [U]are[/U] trying to help you by challenging your apparent assumptions!)

[quote]I'm not saying it's a hardware or windows issue.[/quote]... and we're not claiming that you [U]are[/U] saying that. It's just that nothing you've posted so far rules those out, so we suspect that you're assigning blame to the wrong cause.

[quote]It's not a page file issue, my processing power isn't being taken up by another program, and (if it comes up) I'm not throttling my system. It's not anything like that.[/quote]Fine. Show us your local.txt, prime.txt and worktodo.txt files (after masking out info like passwords, IDs, ComputerGUID, assignment hex strings, & specific exponent numbers) so we can see your setup, and thus have information that might lead us to agree with you!

It's not that we don't understand what you're claiming; it's that you haven't shown us data that convinces us that you're not overlooking a hardware, Windows, or setup cause for your complaints.

[quote]Prime95 is running as fast as it's able to, and it's still leaving unused cycles.[/quote]But, so far, you haven't given us any data that proves it's not a hardware, Windows or setup issue -- Your complaints about performance can be explained by what we've said so far, so we don't see why we're necessarily wrong.

[I]You still haven't posted the information I requested in post #5, which would enable us to see how your setup might be different from what we're assuming or deducing and thus rule out a hardware, Windows or setup explanation![/I]

Cheesehead, I gotta say, your negativity on this forum has definitely increased in the years you've been here.

Something that's been consistently true about this forum is, people just type in the first thing that comes to their head without actually responding to the original comment itself.

I'm definitely not the only one whose computer doesn't use the full processor during stage 2 of P-1 factoring, and your use of a very weak deductive fallacy based on something that is inconsequential to the purpose of the post is beneath even the most frequent ranters of us, and is not relevant to that problem or to the proposed solution since the problem as described exists.

OldCpuSpeed=3200
WorkerThreads=2
Affinity=100
ThreadsPerTest=1
Memory=1100 during 6:40-2:10 else 1100
RollingAverage=951
MaxHighMemWorkers=1

Is it a correct restatement to say that you've been proposing the theory that:

(a) Prime95 is somehow failing to make its normally high use of your system's CPU time during P-1, and that

(b) this is not due to hardware, the operating system, or the way the prime.txt, local.txt, and/or worktodo.txt are set up

?

- - -

[quote=dominicanpapi82;161933]OldCpuSpeed=3200
WorkerThreads=2
Affinity=100
ThreadsPerTest=1
Memory=1100 during 6:40-2:10 else 1100
RollingAverage=951
MaxHighMemWorkers=1[/quote]Okay, explain to me why the evidence you've shown us so far rules out the simple possibility that:

only half of your cores are specified to have Prime95 assignments running on them, which is why your overall system CPU usage is only about 50% even though the P-1 thread uses almost 100% of the core it's running on.

Perhaps I'm mistaken or overlooked something, but I don't see anything you've told us that requires that the answer to your underutilization is any more complicated than that: not enough cores specified to run Prime95 assignments.

What evidence, exactly, in your above posts is incompatible with that possibility?

- - -

One detail I don't yet see: is your system dual-core or quad-core? It seems to me that everything you've posted is compatible with either possibility. (Two threads could be running on one hyperthreaded core, or on both cores of a dual-core system, or on half the cores of a quad-core system.)

Was the system running both TF and P-1 when you took the snapshot in post #6?

- - -

[quote]your negativity on this forum has definitely increased[/quote]Well, gosh, I'm sorry I replied to your opening statement with "No, it takes the operating system to do that." What should I have replied, instead, that wouldn't have been negative but would still have pointed out that your assumption was incorrect? Or am I supposed to agree with statements I know to be mistakes, in order not to be "negative"?

If you think my deductions are fallacious, why don't you present evidence that demonstrates their fallacy instead of merely posting an insult?

[quote=cheesehead;161934]only half of your cores are specified to have Prime95 assignments running on them, which is why your overall system CPU usage is only about 50% even though the P-1 thread uses almost 100% of the core it's running on.[/quote]Make that:

only half of your cores are specified to have Prime95 assignments running on them, which is why P-1 uses only about 50% of your overall system CPU usage, even though the P-1 thread uses almost 100% of the core it's running on.

Let's try something else.

dominicanpapi82,

Do you realize that you've never yet told us in this thread that you've ever seen prime95 use more than 50% of the CPU?

Have you, in fact, ever seen prime95 use more than 50% of the CPU on your system?

If prime.exe does sometimes have a CPU number higher than 50%, what type of assignment(s) is it processing at those times?

Can you show us a screenshot where prime95.exe has a number higher than 50%?

If I understand correctly you have 1 worker on a dual core machine, you can set multithreading to 2 to add a helper thread, if that's what you mean.

I was wrong : P-1 factoring is multithreaded (except during the two GCD phase, but they take about 0,2 % of the time.) Only TF would still be monothreaded. On the other hand assigning four threads to one P-1 test on a quad-core machine leaves about 30 % of the processing power unexploited : see post 94 of the thread [thread=11101]P-1 factoring anyone[/thread] that may contain answers to your interrogations.[QUOTE=dominicanpapi82;161933]WorkerThreads=2
ThreadsPerTest=1
[/QUOTE]Looking at the last data you provide, my theory of one worker with two threads leaving one core idle while doing single threaded tests is wrong. So far, based on the evidence you provided, I must agree with Richard, that you did not provide enough data.

What is you processor ?
When you look at the Prime95 window, are both worker threads active ? You can copy and paste the content of the individual worker windows using the edit menu of Prime95, and edit them to obfuscate all reference to exponents. That way you can post the relevant content here.
Is is possible that you have two assigments both requiring HighMem and no other work queued up for the thread that is now inactive ? You can post the content of your worktodo.txt file after obfuscating the exponents.

Before you give enough data, it is difficult to answer.

Jacob

dominicanpapi82,

I'm sorry that my statement, "... so from our points of view it seems reasonable to think that might not be the only mistaken assumption you're making", was so upsetting.

(I think that's what you meant by "your use of a very weak deductive fallacy based on something that is inconsequential to the purpose of the post". If not, please correct me.)

Writing that you had possibly made a mistaken assumption was never intended as any "accusation", or insult, or demeaning remark by me. To me, making assumptions, and occasionally being mistaken in doing so, is just part of the human condition. Mistaken assumptions are not sins; they're just ... mistakes.

I learned at an early age that everyone makes mistaken assumptions in some way or another. I also learned that most people rarely question their assumptions as often as I did. Later in life, when I started working as a computer programmer, I found that sometimes I found the answer to a problem that other, more-experienced programmers overlooked because they had been mistaken about one or more assumptions they used when trying to solve the problem.

So I began incorporating, as simply a standard step in problem-solving, the conscious questioning of assumptions that I or someone else might be making. That's why I wrote what I did; to me, it was only a standard problem-resolving step (the problem there being that I couldn't figure out why you were so sure all the rest of us were on the wrong track).

You are, and have always been, free to point out what you think anyone else is mistaken of assuming, without that necessarily being considered an insult by the other person ... and indeed that is what you have done by such statements as "I think you misunderstood. I'm not saying there is a big load on my computer taking up processor power from prime95." I took that as a simple suggestion of something I might have been mistakenly assuming. (In fact, I hadn't been assuming that you were saying that; perhaps I should've explicitly said so.)

Later, you wrote [quote]I think you all are looking for a solution to a problem without understanding what I'm saying.

I'm not saying it's a hardware or windows issue. It's not a page file issue, my processing power isn't being taken up by another program, and (if it comes up) I'm not throttling my system. It's not anything like that. Prime95 is running as fast as it's able to, and it's still leaving unused cycles.[/quote]Again, I took that as a suggestion that I (and others) might be making some incorrect assumption in my (our) interpretation of what you had posted so far. So I replied at length to try to explain why those possibilities did not seem likely to me.

Our responses to your posts have been genuine attempts to help. We need more information to figure out the "puzzle" we see about what you've posted.

Perhaps, when this is all resolved, it [I]will[/I] be clear that I and others were mistaken and that you had been correct all along. But right now I (and, I think, others) don't yet understand what our mistake is, so we need your help in illuminating the situation from your side, so that we can understand our mistake and then make progress after correcting our mistake. Conversely, if in fact it turns out that you have also (or instead) made some mistaken assumption, your provision of more data (that you have, but we don't) may help us determine exactly where the barrier to understanding lies.

[QUOTE=dominicanpapi82;161871]
Prime95 gives you the option to work on just one number because, well, you might want to work on just one number. I want to do work on just one number. Not that I need justification because many other people also want to work on just one number, but I want to work on just one number because I am working on a 100M digit mersenne that takes years to finish.

If it's true that P-1 testing can't take full advantage of multiple cores, then given that some multicore computers will be set up to work on just one number, it will definitely be an improvement in throughput (by weeks for 100M digit mersennes) if we are able to simultaneously P-1 test and trial factor the same number. I don't mean with separate code, or some slick engineering to try to save steps if that is at all possible. I mean, just do both on the same number simultaneously just like you could do P-1 testing on one mersenne and trial factoring on another simultaneously. The average PC would save time processing a single number if it simultaneously does P-1 factoring and trial factoring.[/QUOTE]

You are right, prime95 cannot use multiple cores for p-1 factoring of a single number. It only uses multiple cores on a single number for LL testing (first checks or double checks). It is not a bug, it is just what has been implemented so far. You might call it a "known problem" but most people don't consider it a problem.

Yes, it might be possible to notice this is happening and find something else for your idle core to do but it does not seem to be a high priority just now. The 100 million digit stuff you are doing is a bit far out on the radar for now. Maybe when the bulk of the testing gets out there it will be more significant.

You need to realize you are kinda out on the fringe of the project (for now at least) and development efforts have to be triaged since there is a lot of things to do and not all of them can be done at once. We still appreciate your contribution and thanks for pointing out this possible improvement. Just have patience with us while we take care of more pressing issues.

[QUOTE=lfm;164361]You are right, prime95 cannot use multiple cores for p-1 factoring of a single number.[/QUOTE]
Yes, it can. :huh:

[QUOTE=axn;164364]Yes, it can. :huh:[/QUOTE]

Excuse me but if you are right I can't find any documentation supporting your assertion. All I find is from whatsnew.txt for Ver 25.5

5) Supports using multiple logical or physical CPUs on a single LL test.
Except for some hyperthreaded CPUs and only on some FFT sizes, this is
not recommended as you will get more throughput by giving each CPU
its own number to test.

[QUOTE=lfm;164635]Excuse me but if you are right I can't find any documentation supporting your assertion.[/QUOTE]
Hmmm... Perhaps George should update the documentation.

Nonetheless, it is what it is. P-1 supports multiple CPUs. Try it.

EDIT:- Also see post #17

BTW, the correct setting for P-1 to work on two threads is:
[CODE]WorkerThreads=1
ThreadsPerTest=2
[/CODE]

Jumping late into this thread: it looks to me like [i]dominicanpapi82[/i] very likely has a [b]hyperthreaded, single-core[/b] P4-3.2Hz. The apparent "second core" that Windows shows is not really there, it is very very different from the first, physical core. Hyperthreaded CPUs can give a performance boost typically around +10-15% over the non-hyperthreaded single-core, but results vary greatly depending on what the workload is (it could be +30%, it could actually be slower than single-core in some cases). There's plenty of threads on the forum about hyperthreaded performance; you'll probable find some testing results and optimal configurations in there. In the posted screenshot, the system was 42% idle over the doubled cores, which means that useful work is occupying 58/50=116% of the capacity of the non-hyperthreaded singlecore, which looks just about expected. Even if you do get the CPU utilization up to show close to 100%, your throughput is not going to be hugely better (doing simultaneous P-1 and TF will have slightly higher throughput than P-1 alone, but it also means that the P-1 will take nearly twice as long (maybe 1.85x?) to complete compared to it running by itself).

[QUOTE=James Heinrich;164654]Jumping late into this thread: it looks to me like [i]dominicanpapi82[/i] very likely has a [b]hyperthreaded, single-core[/b] P4-3.2Hz. The apparent "second core" that Windows shows is not really there, it is very very different from the first, physical core. Hyperthreaded CPUs can give a performance boost typically around +10-15% over the non-hyperthreaded single-core, but results vary greatly depending on what the workload is (it could be +30%, it could actually be slower than single-core in some cases). There's plenty of threads on the forum about hyperthreaded performance; you'll probable find some testing results and optimal configurations in there. In the posted screenshot, the system was 42% idle over the doubled cores, which means that useful work is occupying 58/50=116% of the capacity of the non-hyperthreaded singlecore, which looks just about expected. Even if you do get the CPU utilization up to show close to 100%, your throughput is not going to be hugely better (doing simultaneous P-1 and TF will have slightly higher throughput than P-1 alone, but it also means that the P-1 will take nearly twice as long (maybe 1.85x?) to complete compared to it running by itself).[/QUOTE]

But wouldn't the OS report both CPUs as active rather than showing 42% idle even tho they were producing less total work results than they might? The idle task will only report time when a CPU finds no other task runnable. It will not and cannot report idleness within instructions due to functional unit conflicts between hyperthreaded CPUs any more than it can report waits for cache misses to go to main memory.

For the idle task to get control and start measuring time you need a task wait such as a page file wait or an intertask lock wait.

Perhaps dominicanpapi82 could tell us exactly what model CPU he has?

P-1 is definitely chewing up almost all of 8 threads here (25.8b4 64-bit, Windows)