Bigger and better GPU sieving drive: Discussion

[frmky]

For both a higher end GTX 480 and a lower end C1060, I got the same speed for the precompiled 2.3 binary and a 3.1 binary I compiled myself. 2.8 Mp/s is closer to what one would expect simply in terms of CUDA cores. 5.1M p/s (336/480) = 3.6M p/s.

[mdettweiler]

Originally Posted by Ken_g6

Edit: Also, have you tested that it's outputting correct results? Testing thread here, and/or test it against the version I compiled.

Confirmed, it does produce all the expected factors in the PPS test range. Should be good to go.

[Karl M Johnson]

Originally Posted by Ken_g6

Can someone with a GTX 470 or 480 (frmky?) confirm whether or not this applies to the higher-end Fermis? I'm guessing that it does, but to a lesser degree.

Since GTX 460 is not sm_20 as GTX 465,470,480, it has somewhat different architecture.
From what I've heard from OpenCL coders, sm_21 GPU's shaders cant fully be used, like all previous GeForce cards. 32 SPs per SM can work as they should, yet other 16 will execute instructions only if it's not dependant on the result of calculation. Like first Pentium, U & V pipes.
Also, sm_21 gpus benefit more from vectorized code than sm_20 do.
And, vectorized code compiled by toolkit 3.2 seems to be 5% faster than compiled by toolkit 3.1, which means NV is improving compiler for sm_21 gpus.

[Ralf Recker]

Originally Posted by Karl M Johnson

Since GTX 460 is not sm_20 as GTX 465,470,480, it has somewhat different architecture.
From what I've heard from OpenCL coders, sm_21 GPU's shaders cant fully be used, like all previous GeForce cards. 32 SPs per SM can work as they should, yet other 16 will execute instructions only if it's not dependant on the result of calculation. Like first Pentium, U & V pipes.
Also, sm_21 gpus benefit more from vectorized code than sm_20 do.
And, vectorized code compiled by toolkit 3.2 seems to be 5% faster than compiled by toolkit 3.1, which means NV is improving compiler for sm_21 gpus.

That would explain the results of a run on a GTX 460 using NVIDIA's profiler:

Occupancy = 0.666667 ( 32 / 48 ) Achieved occupancy = 0.666667 (on 7 SMs)
Occupancy limiting factor = Block-Size

[Ken_g6]

No, occupancy refers to how many blocks are queued up in registers at one time, not how many are being executed at one time. Occupancy only matters to hide latency. Local instruction latency is very low, so it's mostly used to hide memory access latency. There isn't a lot of memory access in this kernel, so it doesn't matter that much.

This is the issue Karl was referring to.

[Ken_g6]

FYI, the latest OpenCL version seems to work fine for most. To use it, you first need to get the ATI Stream SDK and follow these instructions (PDF) to install it. Yes, you have to read that PDF, particularly if you have Linux!

Let me know if you get computation errors or have other problems.

[Ralf Recker]

Originally Posted by Ken_g6

No, occupancy refers to how many blocks are queued up in registers at one time, not how many are being executed at one time. Occupancy only matters to hide latency. Local instruction latency is very low, so it's mostly used to hide memory access latency. There isn't a lot of memory access in this kernel, so it doesn't matter that much.

This is the issue Karl was referring to.

Thanks for the explanation. I remember that I've read this article a few weeks ago and it seems that I've forgotten some details. When I first tested ppsieve-cuda 0.2.0 on the GF104 chip increasing the memory clocks without changing the core/shader clocks didn't have any effect on the runtimes which gave me an indication that memory isn't a limiting factor.

[vaughan]

How do you get the cuda client's DOS box to show the speed it is crunching at? i.e. factors / sec or equivalent nomenclature?

[mdettweiler]

Originally Posted by vaughan

How do you get the cuda client's DOS box to show the speed it is crunching at? i.e. factors / sec or equivalent nomenclature?

It outputs a line like this, which is updated every 60 seconds (by default--this can be changed in ppconfig.txt or on the command line):

p=11233184952321, 2.818M p/sec, 0.05 CPU cores, 23.3% done. ETA 11 Oct 05:10

The p/sec. figure is the most accurate measure of speed (since factors/sec. becomes less useful as factors begin to thin out with the progression of the sieve).

[vaughan]

Interesting, I watched it for 10 minutes and no sign of a speed indication. So I modified the ppconfig.txt using Notepad++ and changed the "Time between status reports" parameter from 60 to 30 then saved. Restarted the application and still no speed displays.

Never mind, GPUZ shows the GTX460 is running at 97-99 percent and I used Ntune to adjust the fan setting from 9 percent to 100 percent and this brought the GPU's temps down from 65C to 55C. The factors file is growing nicely, its at 2331kb so far and we're up to 120233 (range is 120000 - 130000) so that's nearly 25 percent completed in a 2.5 hours.

[gd_barnes]

Originally Posted by vaughan

Interesting, I watched it for 10 minutes and no sign of a speed indication. So I modified the ppconfig.txt using Notepad++ and changed the "Time between status reports" parameter from 60 to 30 then saved. Restarted the application and still no speed displays.

Never mind, GPUZ shows the GTX460 is running at 97-99 percent and I used Ntune to adjust the fan setting from 9 percent to 100 percent and this brought the GPU's temps down from 65C to 55C. The factors file is growing nicely, its at 2331kb so far and we're up to 120233 (range is 120000 - 130000) so that's nearly 25 percent completed in a 2.5 hours.

What range exactly is that? It can't be 12T-13T. It's taking my GPU about 4 days to do a P=1T range. Do you have the correct range plugged in or are you using several GPUs or is my GPU just really slow? :-)

Max, is there a way to cool my GPU? The CPU temps are running kinda high at 75C. I think I'll put an external fan on it.