GPU upgrade

[chris2be8]

Hello,

My GTX 560 Ti has died, one of the fans won't turn (it's stiff when pushed by hand). So I decided to get a new GPU, a GTX 970 which should give a nice speedup.

I've got the new card installed OK, and the system can see it. But I need to upgrade the CUDA version to one that supports it, which may well involve upgrading Linux on the system as well.

If anyone else has a GTX 970 (or similar) running under Linux what CUDA and Linux versions do you use? I'm asking now in case a straightforward upgrade doesn't work.

When I eventually get it working I'll post some figures for how fast it runs ECM compared with the 560.

Chris

[fivemack]

I'll be working this out (under ubuntu-15.10) probably this evening if I'm awake enough - the fans on my GTX580 stopped turning, and the GTX970 arrived yesterday but I spent the evening playing bassoon in the band rather than computer-fettling.

[Mark Rose]

You'll need driver version at least 352.

[chris2be8]

Originally Posted by fivemack

I'll be working this out (under ubuntu-15.10) probably this evening if I'm awake enough - the fans on my GTX580 stopped turning, and the GTX970 arrived yesterday but I spent the evening playing bassoon in the band rather than computer-fettling.

You will probably beat me to it. The system runs Linux Mint 15 which is Ubuntu 13.04 under the covers, and Nvidia don't support a new enough CUDA version on that. So I'll have to upgrade Linux as well (it's due an upgrade anyway).

I'm currently backing up my home dir to another system so I can wipe the system and reinstall Linux if necessary, But the backup probably won't finish this evening (it has lots of small files to copy).

Chris

[jwaltos]

I've been running 970's under Mageia 5; Cuda 6.5 comes with it.
This is one of the simplest distros, in terms of maintenance and use, I have come across.
You can also install directly from Nvidia by choosing the Fedora distro (which will run under Mageia).
I have used 7.0 rc without issue.

[chris2be8]

Originally Posted by chris2be8

When I eventually get it working I'll post some figures for how fast it runs ECM compared with the 560.

And here they are:

chris@4core:~/ggnfs/trunk/tests/m1237_71> time /home/chris/ecm.2741/trunk/ecm -gpu -v -save m1237_71.save 43e6 1 <m1237_71.ini
GMP-ECM 7.0-dev [configured with GMP 5.1.3, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Running on 4core
Input number is 99052130215713425314087719653057747277957246285384978505438516882197068613093361177397394753911498996712838591246881850583816273177651735584539031515037041923006993059932075662627900091316733023818001962039482491 (212 digits)
Using MODMULN [mulredc:0, sqrredc:1]
Computing batch product (of 62034615 bits) of primes below B1=43000000 took 1795ms
GPU: compiled for a NVIDIA GPU with compute capability 2.1.
GPU: will use device 0: GeForce GTX 970, compute capability 5.2, 13 MPs.
GPU: Selection and initialization of the device took 6ms
Using B1=43000000, B2=1, sigma=3:556142369-3:556142784 (416 curves)
dF=0, k=0, d=0, d2=0, i0=0
Expected number of curves to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
621     3658    24066   174580  1382162 1.2e+07 1.1e+08 9.9e+08 5.2e+09 Inf
Computing 416 Step 1 took 592115ms of CPU time / 6418495ms of GPU time
Throughput: 0.065 curves by second (on average 15429.07ms by Step 1)
Expected time to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
2.66h   15.68h  4.30d   31.18d  246.82d 5.79y   53.08y  483.26y 2530y   Inf
Peak memory usage: 37209MB

real    107m0.765s
user    9m53.962s
sys     2m16.343s

The GTX 560 Ti took just over two hours to do 256 curves against the same number. So the GTX 970 is a bit disappointing, I'd hoped for a bigger speedup.

That run was before running make ecm-params but I'd be surprised if that made much difference.

Chris

[Mark Rose]

Originally Posted by chris2be8

The GTX 560 Ti took just over two hours to do 256 curves against the same number. So the GTX 970 is a bit disappointing, I'd hoped for a bigger speedup.

That run was before running make ecm-params but I'd be surprised if that made much difference.

Chris

So it scales like TF scales. The 5xx card are particularly good at that.

[chris2be8]

After running "make ecm-params;./bench_mulredc >> ecm-params.h;make" it sped up to about 1 hour 25 minutes to do stage 1. Which was a pleasant surprise.

I then noticed I'd configured it with --enable-gpu=sm_21 which is for a NVIDIA GPU of compute capability 2.1. But the 970 should support compute capability 3.0 if I configure ecm with --enable-gpu=sm_30. I've rebuilt ecm with that, but not tested it yet. Once I've finished ecm prep for 1237^71-1 (ETA tomorrow morning) I'll try out CC 3.0.

The delay is that I need a quiet system to run make ecm-params;./bench_mulredc >> ecm-params.h. Does that actually use the GPU? If not the current ecm-params.h would be fine.

Chris

[chris2be8]

I was about to rebuild ecm with --enable-gpu=sm_30.when I looked at the source for the configure script and found it also supports --enable-gpu=sm_35. So I built ecm with that enabled. But the result was disappointing:

chris@4core:~/ggnfs/trunk/tests/m1237_71> \time -v /home/chris/ecm.sm35/trunk/ecm -gpu -v -save m1237_71.save 43e6 1 <m1237_71.ini
GMP-ECM 7.0-dev [configured with GMP 5.1.3, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Running on 4core
Input number is 99052130215713425314087719653057747277957246285384978505438516882197068613093361177397394753911498996712838591246881850583816273177651735584539031515037041923006993059932075662627900091316733023818001962039482491 (212 digits)
Using MODMULN [mulredc:1, sqrredc:1]
Computing batch product (of 62034615 bits) of primes below B1=43000000 took 1802ms
GPU: compiled for a NVIDIA GPU with compute capability 3.5.
GPU: will use device 0: GeForce GTX 970, compute capability 5.2, 13 MPs.
GPU: Selection and initialization of the device took 9ms
Using B1=43000000, B2=1, sigma=3:4053799753-3:4053800584 (832 curves)
dF=0, k=0, d=0, d2=0, i0=0
Expected number of curves to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
621     3658    24066   174580  1382162 1.2e+07 1.1e+08 9.9e+08 5.2e+09 Inf
Computing 832 Step 1 took 653067ms of CPU time / 11675476ms of GPU time
Throughput: 0.071 curves by second (on average 14033.02ms by Step 1)
Expected time to find a factor of n digits:
35      40      45      50      55      60      65      70      75      80
2.42h   14.26h  3.91d   28.36d  224.49d 5.27y   48.27y  439.54y 2301y   Inf
Peak memory usage: 37209MB
        Command being timed: "/home/chris/ecm.sm35/trunk/ecm -gpu -v -save m1237_71.save 43e6 1"
        User time (seconds): 655.03
        System time (seconds): 101.24
        Percent of CPU this job got: 6%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 3:14:37
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 120960
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 17657
        Voluntary context switches: 5945503
        Involuntary context switches: 19005
        Swaps: 0
        File system inputs: 0
        File system outputs: 912
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

So although it did twice as many curves as with compute capability 2.1 it's still slower per curve than 2.1 was after I'd run make ecm-params.

When I have some free GPU time I'll try running the tuned version for CC 2.1 exactly as above. I can't see how running ecm under time or adding -v could slow it significantly, but I'll check to be sure.

Chris