Msieve 1.53 feedback - Page 2

wombatman · 2014-08-06, 14:33

Absolutely! Please let me know if there are any dlls missing.

pomodoro · 2014-08-23, 15:26

Hi,
Using I7 4790K + GTX770 ang your cuda Msieve 1.53 , take a look:
What´s wrong ?

Number: example
N = 2881039827457895971881627053137530734638790825166127496066674320241571446494762386620442953820735453 (100 digits)
Divisors found:
r1=618162834186865969389336374155487198277265679 (pp45)
r2=4660648728983566373964395375209529291596595400646068307 (pp55)
Version: Msieve v. 1.53 (SVN unknown)
Total time: 9.04 hours.
Factorization parameters were as follows:
n: 2881039827457895971881627053137530734638790825166127496066674320241571446494762386620442953820735453
Y0: -1191805077826652345824255
Y1: 1949275902691
c0: -900094273514840852683747752
c1: -7337844764575786222070
c2: -3360162038991689
c3: 258820560
c4: 1428
skew: 1641627.64
type: gnfs
Factor base limits: 1800000/1800000
Large primes per side: 3
Large prime bits: 26/26
Sieved algebraic special-q in [0, 0)
Total raw relations: 5187509
Relations: 251318 relations
Pruned matrix : 155852 x 156079
Polynomial selection time: 7.08 hours.
Total sieving time: 1.90 hours.
Total relation processing time: 0.02 hours.
Matrix solve time: 0.03 hours.
time per square root: 0.01 hours.
Prototype def-par.txt line would be: gnfs,99,4,58,1500,0.003,0.4,220,15,10000,2000,1800000,1800000,26,26,48,48,2.5,2.5,100000
total time: 9.04 hours.
Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
Windows-7-6.1.7601-SP1
processors: 2, speed: 4.00GHz

pomodoro · 2014-08-23, 15:32

Hi,
May you take a look in my log using your cuda Msieve 1.53
What´s wrong ?

wombatman · 2014-08-23, 19:58

Looks like everything is fine to me. The polynomial search took ~7 hours and the sieving, matrix-building, and square root steps took ~2 hours.

If you want to cut down the polynomial time, you can set your own stage 1 and stage 2 norms. The empirical rule found on here was to set the stage 1 at the default stage 1 value (which varies depending on the number being factored) divided by 10. Otherwise, though, everything seems fine.

VBCurtis · 2014-08-23, 20:32

You could/should have stopped polynomial search after no more than 10 minutes; the other steps would have been a little bit longer, but you still would have finished in under 3 hours.

I disagree with wombatman- I think stage1 norm should be left alone, but stage2norm should be divided by 20 or so when using CUDA for poly select. It makes no difference at all for a project this small, but when poly select runs for days rather than minutes (say, for 150-160 digit problems) it helps to alter parameters a bit here and there.

It's generally wise to work your way up from under 120 digits to the 140+ range slowly, say doing a dozen or so projects among the 120-140 range to see how parameters change and how the various flags work. Often, experimenting on a ~125 digit number will teach you possible ways to save hours or even days on 150+ digit projects.

wombatman · 2014-08-23, 23:05

Thanks for the correction/suggestion, VBCurtis. That'll help me with some of my polynomial searching.

wombatman · 2014-09-07, 04:00

Quick question about the memory usage for the linear algebra steps. On my current project, the matrix is ~15.9M big. The RAM usage has been, at various times during the actual linear algebra completion step (so not during the duplicate removal or relations cleanup) between 7GB and 11GB. Is this to be expected?

debrouxl · 2014-09-07, 09:34

Just a wild guess: did these bursts of higher memory consumption occur while a checkpoint was being made ?

wombatman · 2014-09-07, 20:01

Honestly, I couldn't tell you for sure. When I initially started the run, the memory usage started high (about 11GB) and then gradually dropped over time to somewhere in the neighborhood of 7-8GB. Then my computer crashed, and when I restarted, the memory usage was back to about 11GB.

jasonp · 2014-09-08, 00:14

Building the matrix for the first time is highly memory intensive, so it makes sense that a matrix build increases the memory use. I'd be somewhat surprised if restarting from a checkpoint also had high memory use, since the matrix in that case is already built and just needs to be read from disk. It's also possible that the files to read from are also left in disk cache by the OS, and never really used afterwards so they are slowly replaced by pages from other programs.

wombatman · 2014-09-08, 00:50

That would make some sense. Thanks!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve 1.50 feedback	firejuggler	Msieve	99	2013-02-17 11:53
Msieve v1.48 feedback	Jeff Gilchrist	Msieve	48	2011-06-10 18:18
Msieve 1.43 feedback	Jeff Gilchrist	Msieve	47	2009-11-24 15:53
Msieve 1.42 feedback	Andi47	Msieve	167	2009-10-18 19:37
Msieve 1.41 Feedback	Batalov	Msieve	130	2009-06-09 16:01

2014-08-23, 19:58	#15
wombatman I moo ablest echo power! May 2013 29·61 Posts	Looks like everything is fine to me. The polynomial search took ~7 hours and the sieving, matrix-building, and square root steps took ~2 hours. If you want to cut down the polynomial time, you can set your own stage 1 and stage 2 norms. The empirical rule found on here was to set the stage 1 at the default stage 1 value (which varies depending on the number being factored) divided by 10. Otherwise, though, everything seems fine.

2014-08-23, 20:32	#16
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	You could/should have stopped polynomial search after no more than 10 minutes; the other steps would have been a little bit longer, but you still would have finished in under 3 hours. I disagree with wombatman- I think stage1 norm should be left alone, but stage2norm should be divided by 20 or so when using CUDA for poly select. It makes no difference at all for a project this small, but when poly select runs for days rather than minutes (say, for 150-160 digit problems) it helps to alter parameters a bit here and there. It's generally wise to work your way up from under 120 digits to the 140+ range slowly, say doing a dozen or so projects among the 120-140 range to see how parameters change and how the various flags work. Often, experimenting on a ~125 digit number will teach you possible ways to save hours or even days on 150+ digit projects.

2014-08-23, 23:05	#17
wombatman I moo ablest echo power! May 2013 29·61 Posts	Thanks for the correction/suggestion, VBCurtis. That'll help me with some of my polynomial searching.

2014-09-07, 04:00	#18
wombatman I moo ablest echo power! May 2013 29·61 Posts	Quick question about the memory usage for the linear algebra steps. On my current project, the matrix is ~15.9M big. The RAM usage has been, at various times during the actual linear algebra completion step (so not during the duplicate removal or relations cleanup) between 7GB and 11GB. Is this to be expected?

2014-09-07, 09:34	#19
debrouxl Sep 2009 977 Posts	Just a wild guess: did these bursts of higher memory consumption occur while a checkpoint was being made ?

2014-09-07, 20:01	#20
wombatman I moo ablest echo power! May 2013 11011101001₂ Posts	Honestly, I couldn't tell you for sure. When I initially started the run, the memory usage started high (about 11GB) and then gradually dropped over time to somewhere in the neighborhood of 7-8GB. Then my computer crashed, and when I restarted, the memory usage was back to about 11GB.

2014-09-08, 00:14	#21
jasonp Tribal Bullet Oct 2004 3,541 Posts	Building the matrix for the first time is highly memory intensive, so it makes sense that a matrix build increases the memory use. I'd be somewhat surprised if restarting from a checkpoint also had high memory use, since the matrix in that case is already built and just needs to be read from disk. It's also possible that the files to read from are also left in disk cache by the OS, and never really used afterwards so they are slowly replaced by pages from other programs.

2014-09-08, 00:50	#22
wombatman I moo ablest echo power! May 2013 6E9₁₆ Posts	That would make some sense. Thanks!