GPU upgrade

[chris2be8]

Hello,

My GTX 560 Ti has died, one of the fans won't turn (it's stiff when pushed by hand). So I decided to get a new GPU, a GTX 970 which should give a nice speedup.

I've got the new card installed OK, and the system can see it. But I need to upgrade the CUDA version to one that supports it, which may well involve upgrading Linux on the system as well.

If anyone else has a GTX 970 (or similar) running under Linux what CUDA and Linux versions do you use? I'm asking now in case a straightforward upgrade doesn't work.

When I eventually get it working I'll post some figures for how fast it runs ECM compared with the 560.

Chris

[fivemack]

I'll be working this out (under ubuntu-15.10) probably this evening if I'm awake enough - the fans on my GTX580 stopped turning, and the GTX970 arrived yesterday but I spent the evening playing bassoon in the band rather than computer-fettling.

[Mark Rose]

You'll need driver version at least 352.

[chris2be8]

Originally Posted by fivemack

I'll be working this out (under ubuntu-15.10) probably this evening if I'm awake enough - the fans on my GTX580 stopped turning, and the GTX970 arrived yesterday but I spent the evening playing bassoon in the band rather than computer-fettling.

You will probably beat me to it. The system runs Linux Mint 15 which is Ubuntu 13.04 under the covers, and Nvidia don't support a new enough CUDA version on that. So I'll have to upgrade Linux as well (it's due an upgrade anyway).

I'm currently backing up my home dir to another system so I can wipe the system and reinstall Linux if necessary, But the backup probably won't finish this evening (it has lots of small files to copy).

Chris

I've been running 970's under Mageia 5; Cuda 6.5 comes with it.
This is one of the simplest distros, in terms of maintenance and use, I have come across.
You can also install directly from Nvidia by choosing the Fedora distro (which will run under Mageia).
I have used 7.0 rc without issue.

[chris2be8]

Originally Posted by chris2be8

When I eventually get it working I'll post some figures for how fast it runs ECM compared with the 560.

And here they are:

chris@4core:~/ggnfs/trunk/tests/m1237_71> time /home/chris/ecm.2741/trunk/ecm -gpu -v -save m1237_71.save 43e6 1 <m1237_71.ini
GMP-ECM 7.0-dev [configured with GMP 5.1.3, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Running on 4core
Input number is 99052130215713425314087719653057747277957246285384978505438516882197068613093361177397394753911498996712838591246881850583816273177651735584539031515037041923006993059932075662627900091316733023818001962039482491 (212 digits)
Using MODMULN [mulredc:0, sqrredc:1]
Computing batch product (of 62034615 bits) of primes below B1=43000000 took 1795ms
GPU: compiled for a NVIDIA GPU with compute capability 2.1.
GPU: will use device 0: GeForce GTX 970, compute capability 5.2, 13 MPs.
GPU: Selection and initialization of the device took 6ms
Using B1=43000000, B2=1, sigma=3:556142369-3:556142784 (416 curves)
dF=0, k=0, d=0, d2=0, i0=0
Expected number of curves to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
621     3658    24066   174580  1382162 1.2e+07 1.1e+08 9.9e+08 5.2e+09 Inf
Computing 416 Step 1 took 592115ms of CPU time / 6418495ms of GPU time
Throughput: 0.065 curves by second (on average 15429.07ms by Step 1)
Expected time to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
2.66h   15.68h  4.30d   31.18d  246.82d 5.79y   53.08y  483.26y 2530y   Inf
Peak memory usage: 37209MB

real    107m0.765s
user    9m53.962s
sys     2m16.343s

The GTX 560 Ti took just over two hours to do 256 curves against the same number. So the GTX 970 is a bit disappointing, I'd hoped for a bigger speedup.

That run was before running make ecm-params but I'd be surprised if that made much difference.

Chris

[Mark Rose]

Originally Posted by chris2be8

The GTX 560 Ti took just over two hours to do 256 curves against the same number. So the GTX 970 is a bit disappointing, I'd hoped for a bigger speedup.

That run was before running make ecm-params but I'd be surprised if that made much difference.

Chris

So it scales like TF scales. The 5xx card are particularly good at that.

[chris2be8]

After running "make ecm-params;./bench_mulredc >> ecm-params.h;make" it sped up to about 1 hour 25 minutes to do stage 1. Which was a pleasant surprise.

I then noticed I'd configured it with --enable-gpu=sm_21 which is for a NVIDIA GPU of compute capability 2.1. But the 970 should support compute capability 3.0 if I configure ecm with --enable-gpu=sm_30. I've rebuilt ecm with that, but not tested it yet. Once I've finished ecm prep for 1237^71-1 (ETA tomorrow morning) I'll try out CC 3.0.

The delay is that I need a quiet system to run make ecm-params;./bench_mulredc >> ecm-params.h. Does that actually use the GPU? If not the current ecm-params.h would be fine.

Chris

[chris2be8]

I was about to rebuild ecm with --enable-gpu=sm_30.when I looked at the source for the configure script and found it also supports --enable-gpu=sm_35. So I built ecm with that enabled. But the result was disappointing:

chris@4core:~/ggnfs/trunk/tests/m1237_71> \time -v /home/chris/ecm.sm35/trunk/ecm -gpu -v -save m1237_71.save 43e6 1 <m1237_71.ini
GMP-ECM 7.0-dev [configured with GMP 5.1.3, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Running on 4core
Input number is 99052130215713425314087719653057747277957246285384978505438516882197068613093361177397394753911498996712838591246881850583816273177651735584539031515037041923006993059932075662627900091316733023818001962039482491 (212 digits)
Using MODMULN [mulredc:1, sqrredc:1]
Computing batch product (of 62034615 bits) of primes below B1=43000000 took 1802ms
GPU: compiled for a NVIDIA GPU with compute capability 3.5.
GPU: will use device 0: GeForce GTX 970, compute capability 5.2, 13 MPs.
GPU: Selection and initialization of the device took 9ms
Using B1=43000000, B2=1, sigma=3:4053799753-3:4053800584 (832 curves)
dF=0, k=0, d=0, d2=0, i0=0
Expected number of curves to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
621     3658    24066   174580  1382162 1.2e+07 1.1e+08 9.9e+08 5.2e+09 Inf
Computing 832 Step 1 took 653067ms of CPU time / 11675476ms of GPU time
Throughput: 0.071 curves by second (on average 14033.02ms by Step 1)
Expected time to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
2.42h   14.26h  3.91d   28.36d  224.49d 5.27y   48.27y  439.54y 2301y   Inf
Peak memory usage: 37209MB
        Command being timed: "/home/chris/ecm.sm35/trunk/ecm -gpu -v -save m1237_71.save 43e6 1"
        User time (seconds): 655.03
        System time (seconds): 101.24
        Percent of CPU this job got: 6%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 3:14:37
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 120960
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 17657
        Voluntary context switches: 5945503
        Involuntary context switches: 19005
        Swaps: 0
        File system inputs: 0
        File system outputs: 912
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

So although it did twice as many curves as with compute capability 2.1 it's still slower per curve than 2.1 was after I'd run make ecm-params.

When I have some free GPU time I'll try running the tuned version for CC 2.1 exactly as above. I can't see how running ecm under time or adding -v could slow it significantly, but I'll check to be sure.

Chris