CUDA Tutorials/Learning with an eye to large numbers

wombatman · 2014-04-04, 18:27

I have a good amount of free time on my hands right now, and I would like to start poking around in CUDA programming, but I'm having some trouble finding good beginner's tutorials. I understand the general process of "Prepare some data, move it to the GPU, do a lot of repetitive tasks, move the results back to the host" but I'd like some more in-depth teaching about what kinds of tasks are good for parallel processes.

I'd also like to learn how to handle large numbers like those used with the GPU implementation of GMP-ECM and things like LLR-CUDA and the like.

Can anyone provide suggestions (either books or online) where I might get a good start? Thanks!

Batalov · 2014-04-04, 18:45

You might want to decouple the two parts of this: CUDA and large numbers.

Both are challenging and in many ways independent (it might be surprising if there exists a book on both at the same time). You can start CUDA separately, and then maybe do simple steps, like check how tf_72bit.cu works (in the mfaktc package, the smallest tier); it might be tough at first, but if you like the "learn to swim by stepping off the boat" paradigm, you might enjoy that. ;-)

wombatman · 2014-04-04, 19:04

For better or worse, I definitely tend toward the boat-jumping ;) Thanks for the mfaktc suggestion. I'll go cross my eyes at that for a while.

I also found a nice free online course from udacity. I'm only a little bit in, but it seems like they're going to start with a nice breakdown of how the GPU hardware is set up for parallelization and then (I hope) get into actual code as well.

Udacity course is here: https://www.udacity.com/course/cs344 (Title is Intro to Parallel Programming with CUDA).

chalsall · 2014-04-04, 19:06

Quote:

Originally Posted by Batalov

... it might be tough at first, but if you like the "learn to swim by stepping off the boat" paradigm, you might enjoy that. ;-)

The best way to learn, IMHO.

ET_ · 2014-04-05, 08:36

Quote:

Originally Posted by wombatman

For better or worse, I definitely tend toward the boat-jumping ;) Thanks for the mfaktc suggestion. I'll go cross my eyes at that for a while.

I also found a nice free online course from udacity. I'm only a little bit in, but it seems like they're going to start with a nice breakdown of how the GPU hardware is set up for parallelization and then (I hope) get into actual code as well.

Udacity course is here: https://www.udacity.com/course/cs344 (Title is Intro to Parallel Programming with CUDA).

I'm doing that course as well

I'm stuck at the reduction lessons after the tone-mapping, trying to understand how to perform a radix-sort on multi-block data (the exercise is on red-eye removal).

Let me know if you have hints...

Luigi

wombatman · 2014-04-05, 16:22

I don't think I've gotten to the red-eye removal yet, but I'll let you know if I can get it worked out.

R.D. Silverman · 2014-04-07, 12:35

Quote:

Originally Posted by wombatman

I'd also like to learn how to handle large numbers like those used with the GPU implementation of GMP-ECM and things like LLR-CUDA and the like.

Knuth, TAOCP, Vol II.

wombatman · 2014-04-07, 14:53

Excellent! Thank you!

Quote:

Originally Posted by R.D. Silverman

Knuth, TAOCP, Vol II.

tServo · 2014-04-10, 16:59

Dr Dobbs ( www.drdobbs.com ) had a series a year or 2 ago by Rob Farber about learning Cuda. I don't know how it compares to the other udacity course. It was titled something like " Supercomputing for the Masses".

wombatman · 2014-04-10, 21:25

Another great find, thanks!

Speaking of Knuth and TAOCP, here's a section from an NVIDIA-provided CUDA header defining a double double precision type:

Code:

/* Compute error-free sum of two unordered doubles. See Knuth, TAOCP vol. 2 */
__device__ __forceinline__ dbldbl add_double_to_dbldbl (double a, double b)
{
    double t1, t2;
    dbldbl z;
    z.y = __dadd_rn (a, b);
    t1 = __dadd_rn (z.y, -a);
    t2 = __dadd_rn (z.y, -t1);
    t1 = __dadd_rn (b, -t1);
    t2 = __dadd_rn (a, -t2);
    z.x = __dadd_rn (t1, t2);
    return z;
}

Just thought that was neat.

Ken_g6 · 2014-04-10, 23:29

A lot depends on "How large is 'large'?". And by that I mean the numbers actually being worked with. You can sieve or trial-factor extremely large numbers without using numbers larger than the factors you're trying.

If "large" is 64 bits or less, check out my multi-K-and-N siever.

If "large" is about 65-96 bits, look at mfaktc.

If "large" is really a lot larger than 96 bits...maybe look at CUDALucas?

2014-04-04, 18:27	#1
wombatman I moo ablest echo power! May 2013 29·61 Posts	CUDA Tutorials/Learning with an eye to large numbers I have a good amount of free time on my hands right now, and I would like to start poking around in CUDA programming, but I'm having some trouble finding good beginner's tutorials. I understand the general process of "Prepare some data, move it to the GPU, do a lot of repetitive tasks, move the results back to the host" but I'd like some more in-depth teaching about what kinds of tasks are good for parallel processes. I'd also like to learn how to handle large numbers like those used with the GPU implementation of GMP-ECM and things like LLR-CUDA and the like. Can anyone provide suggestions (either books or online) where I might get a good start? Thanks!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Discussion of Large Numbers	Merfighters	Miscellaneous Math	2	2010-10-29 16:51
Calculating large numbers	Historian	Information & Answers	4	2010-03-26 19:39
Squaring large Numbers	SQUARE	Information & Answers	7	2009-05-10 09:13
Maths Tutorials -I	devarajkandadai	Miscellaneous Math	0	2004-12-16 02:53
How do I get LARGE numbers	Bundu	Software	5	2004-08-26 01:56

2014-04-04, 18:45	#2
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 3⁶·13 Posts	You might want to decouple the two parts of this: CUDA and large numbers. Both are challenging and in many ways independent (it might be surprising if there exists a book on both at the same time). You can start CUDA separately, and then maybe do simple steps, like check how tf_72bit.cu works (in the mfaktc package, the smallest tier); it might be tough at first, but if you like the "learn to swim by stepping off the boat" paradigm, you might enjoy that. ;-)

2014-04-04, 19:04	#3
wombatman I moo ablest echo power! May 2013 29·61 Posts	For better or worse, I definitely tend toward the boat-jumping ;) Thanks for the mfaktc suggestion. I'll go cross my eyes at that for a while. I also found a nice free online course from udacity. I'm only a little bit in, but it seems like they're going to start with a nice breakdown of how the GPU hardware is set up for parallelization and then (I hope) get into actual code as well. Udacity course is here: https://www.udacity.com/course/cs344 (Title is Intro to Parallel Programming with CUDA).

2014-04-05, 16:22	#6
wombatman I moo ablest echo power! May 2013 29·61 Posts	I don't think I've gotten to the red-eye removal yet, but I'll let you know if I can get it worked out.

2014-04-10, 16:59	#9
tServo "Marv" May 2009 near the Tannhäuser Gate 2×3×109 Posts	Dr Dobbs ( www.drdobbs.com ) had a series a year or 2 ago by Rob Farber about learning Cuda. I don't know how it compares to the other udacity course. It was titled something like " Supercomputing for the Masses".

2014-04-10, 23:29	#11
Ken_g6 Jan 2005 Caught in a sieve 5·79 Posts	A lot depends on "How large is 'large'?". And by that I mean the numbers actually being worked with. You can sieve or trial-factor extremely large numbers without using numbers larger than the factors you're trying. If "large" is 64 bits or less, check out my multi-K-and-N siever. If "large" is about 65-96 bits, look at mfaktc. If "large" is really a lot larger than 96 bits...maybe look at CUDALucas?