[QUOTE=jbristow;121180]I've been getting errors trying to use 4-threaded Lanczos using Msieve 1.32 on my quad core. I just compiled a 64-bit version using Visual Studio 2008 tonight, so that may be relevant. The rest of the filtering has been working fine. I'm going to try 2 threads overnight to see if that works. Thanks.
[/QUOTE]
Not sure what the issue is; it isn't the assembly code since that's all turned off for 64-bit MSVC. The official binary is compiled with MinGW, which uses win32 threads already, and I didn't see a problem in local testing. Keep me posted.

jasonp

my dad managed to compile a 32 bit version of msieve 1.26 using Visual Studio 2005

I have a similar error with 2 threads. It restarted, but since it's now 20% complete I'll let it keep running. If it restarts again I'll try single-threaded.

[code]
Fri Dec 21 00:24:38 2007 commencing linear algebra
Fri Dec 21 00:24:40 2007 read 3940821 cycles
Fri Dec 21 00:24:53 2007 cycles contain 10791573 unique relations
Fri Dec 21 00:26:33 2007 read 10791573 relations
Fri Dec 21 00:26:50 2007 using 32 quadratic characters above 268435130
Fri Dec 21 00:32:03 2007 read 3940821 cycles
Fri Dec 21 00:32:10 2007 matrix is 3940621 x 3940821 with weight 357751078 (avg 90.78/col)
Fri Dec 21 00:32:22 2007 filtering completed in 1 passes
Fri Dec 21 00:32:23 2007 matrix is 3940621 x 3940821 with weight 357751078 (avg 90.78/col)
Fri Dec 21 00:32:46 2007 read 3940821 cycles
Fri Dec 21 00:32:54 2007 matrix is 3940621 x 3940821 with weight 357751078 (avg 90.78/col)
Fri Dec 21 00:32:54 2007 saving the first 48 matrix rows for later
Fri Dec 21 00:32:57 2007 matrix is 3940573 x 3940821 with weight 273207890 (avg 69.33/col)
Fri Dec 21 00:32:57 2007 matrix includes 64 packed rows
Fri Dec 21 00:32:57 2007 using block size 65536 for processor cache size 4096 kB
Fri Dec 21 00:33:14 2007 commencing Lanczos iteration (2 threads)
Fri Dec 21 02:13:07 2007 lanczos error: submatrix is not invertible
Fri Dec 21 02:13:07 2007 lanczos error (dim = 215799): not all columns used
Fri Dec 21 02:13:07 2007 lanczos halted after 3413 iterations (dim = 215799)
Fri Dec 21 02:13:07 2007 linear algebra failed; retrying...
Fri Dec 21 02:13:07 2007 commencing Lanczos iteration (2 threads)
[/code]

[QUOTE=jbristow;121205]I have a similar error with 2 threads. It restarted, but since it's now 20% complete I'll let it keep running. If it restarts again I'll try single-threaded.
[/QUOTE]
Does the official binary work in 64-bit windows? If you have to restart anyway and you have the time, could you restart with the official binary as well? It can certainly wait until after the factorization completes. If you have to switch to single-threaded and it still fails, then it's something about the assembly code and we can work from there.

[quote=jasonp;121207]Does the official binary work in 64-bit windows? If you have to restart anyway and you have the time, could you restart with the official binary as well? It can certainly wait until after the factorization completes. If you have to switch to single-threaded and it still fails, then it's something about the assembly code and we can work from there.[/quote]

I've never tried multi-threaded with the official Win32 binary. The Win32 binary can't do the Lanczos on this number due to memory constraints, but I can try another smaller factorization multi-threaded.

The Win64 2-thread run I let continue this morning is now 62% done (the restart was only 5% in).

I've done lots of single-threaded runs with the official Win32 binary in 64-bit Vista and have had no similar issues.

This morning it was 107% done and still going, so I stopped it and will give single-threaded a try.

My single-threaded run using the 64-bit msieve I tried to compile failed, so this is probably just an issue with my lack of experience in compiling. I went back to poking around in the code, and saw some "_WIN64" ifdefs around that weren't being triggered, so I added that to the preprocessing and recompiled. It looks like they determine whether or not to use assembler. Is it plausible that could have caused my earlier problems?

[QUOTE=jbristow;121402]My single-threaded run using the 64-bit msieve I tried to compile failed, so this is probably just an issue with my lack of experience in compiling. I went back to poking around in the code, and saw some "_WIN64" ifdefs around that weren't being triggered, so I added that to the preprocessing and recompiled. It looks like they determine whether or not to use assembler. Is it plausible that could have caused my earlier problems?[/QUOTE]
I've used MSVC but never with msieve (the build project requires a more advanced version than I ve had access to) and never in 64-bit. Brian Gladman maintains the build project, so if it needs changes I suggest you coordinate with him. I also suggest you use a smaller size problem to test your changes, since I know from hard experience that trying one change and then waiting days to see if it worked will get old real fast.

Jason, sorry for running around in circles here.

I took your advice and tried testing a smaller number (and ignored my compiled version to limit another variable). Using the 32-bit binary off of the website and 4 threads I found a similar problem in Vista x64. I stopped the linear algebra at 134.5% complete. I've attached the log below.

I did complete a single-threaded NFS run using the Win32 v1.32 binary on 12-20. I'll try 2-threaded and another single-threaded run. Msieve v1.32 has been successfully completing qs runs the last few days (single-threaded) so if it's a hardware issue it's only been showing up here. If this isn't reproducible, I can run some system tests to try to rule that out.

Thanks.

[code]
Mon Dec 24 13:31:00 2007 Msieve v. 1.32
Mon Dec 24 13:31:00 2007 random seeds: 4d037aa8 b54b1477
Mon Dec 24 13:31:00 2007 factoring 231263923457647266191201900825138357653841214261862503299078298545422418656633564873057148712169693291689092000371140172272582039303233 (135 digits)
Mon Dec 24 13:31:01 2007 searching for 15-digit factors
Mon Dec 24 13:31:03 2007 commencing number field sieve (135-digit input)
Mon Dec 24 13:31:03 2007 R0: -80332590188508258239849249537
Mon Dec 24 13:31:03 2007 R1: 9354238358105289311446368256
Mon Dec 24 13:31:03 2007 A0: -1
Mon Dec 24 13:31:03 2007 A1: -3
Mon Dec 24 13:31:03 2007 A2: 6
Mon Dec 24 13:31:03 2007 A3: 4
Mon Dec 24 13:31:03 2007 A4: -5
Mon Dec 24 13:31:03 2007 A5: -1
Mon Dec 24 13:31:03 2007 A6: 1
Mon Dec 24 13:31:03 2007 size score = 3.603096e-008, Murphy alpha = 3.078948, combined = 1.494952e-008
Mon Dec 24 13:31:15 2007 restarting with 13777371 relations
Mon Dec 24 13:31:15 2007 generating factor base
Mon Dec 24 13:31:20 2007 factor base complete:
Mon Dec 24 13:31:20 2007 476648 rational roots (max prime = 6999997)
Mon Dec 24 13:31:20 2007 476539 algebraic roots (max prime = 6999929)
Mon Dec 24 13:31:21 2007 added 79423 free relations
Mon Dec 24 13:31:21 2007
Mon Dec 24 13:31:21 2007 commencing relation filtering
Mon Dec 24 13:31:21 2007 commencing duplicate removal, pass 1
Mon Dec 24 13:33:02 2007 found 1498656 hash collisions in 13856794 relations
Mon Dec 24 13:33:02 2007 commencing duplicate removal, pass 2
Mon Dec 24 13:33:26 2007 found 1205165 duplicates and 12651629 unique relations
Mon Dec 24 13:33:26 2007 memory use: 65.3 MB
Mon Dec 24 13:33:31 2007 ignoring smallest 536962 rational and 536413 algebraic ideals
Mon Dec 24 13:33:31 2007 filtering ideals above 7955151
Mon Dec 24 13:33:31 2007 need 1824737 more relations than ideals
Mon Dec 24 13:33:31 2007 commencing singleton removal, pass 1
Mon Dec 24 13:35:11 2007 relations with 0 large ideals: 138073
Mon Dec 24 13:35:11 2007 relations with 1 large ideals: 584566
Mon Dec 24 13:35:11 2007 relations with 2 large ideals: 2192817
Mon Dec 24 13:35:11 2007 relations with 3 large ideals: 4134331
Mon Dec 24 13:35:11 2007 relations with 4 large ideals: 3988482
Mon Dec 24 13:35:11 2007 relations with 5 large ideals: 1613360
Mon Dec 24 13:35:11 2007 relations with 6 large ideals: 0
Mon Dec 24 13:35:11 2007 relations with 7+ large ideals: 0
Mon Dec 24 13:35:11 2007 12651629 relations and about 10684348 large ideals
Mon Dec 24 13:35:11 2007 commencing singleton removal, pass 2
Mon Dec 24 13:36:53 2007 found 3585547 singletons
Mon Dec 24 13:36:53 2007 current dataset: 9066082 relations and about 6661211 large ideals
Mon Dec 24 13:36:53 2007 commencing singleton removal, pass 3
Mon Dec 24 13:38:10 2007 found 793843 singletons
Mon Dec 24 13:38:10 2007 current dataset: 8272239 relations and about 5838031 large ideals
Mon Dec 24 13:38:10 2007 commencing singleton removal, pass 4
Mon Dec 24 13:39:21 2007 found 191911 singletons
Mon Dec 24 13:39:21 2007 current dataset: 8080328 relations and about 5644234 large ideals
Mon Dec 24 13:39:21 2007 commencing singleton removal, final pass
Mon Dec 24 13:40:35 2007 memory use: 341.3 MB
Mon Dec 24 13:40:35 2007 commencing in-memory singleton removal
Mon Dec 24 13:40:36 2007 begin with 8080328 relations and 5991982 unique ideals
Mon Dec 24 13:40:44 2007 reduce to 7472201 relations and 5374906 ideals in 13 passes
Mon Dec 24 13:40:44 2007 max relations containing the same ideal: 52
Mon Dec 24 13:40:47 2007 removing 842694 relations and 706415 ideals in 136279 cliques
Mon Dec 24 13:40:48 2007 commencing in-memory singleton removal
Mon Dec 24 13:40:48 2007 begin with 6629507 relations and 5374906 unique ideals
Mon Dec 24 13:40:52 2007 reduce to 6575133 relations and 4612992 ideals in 7 passes
Mon Dec 24 13:40:52 2007 max relations containing the same ideal: 48
Mon Dec 24 13:40:55 2007 removing 632705 relations and 496426 ideals in 136279 cliques
Mon Dec 24 13:40:55 2007 commencing in-memory singleton removal
Mon Dec 24 13:40:55 2007 begin with 5942428 relations and 4612992 unique ideals
Mon Dec 24 13:40:59 2007 reduce to 5905208 relations and 4078637 ideals in 7 passes
Mon Dec 24 13:40:59 2007 max relations containing the same ideal: 43
Mon Dec 24 13:41:01 2007 removing 14706 relations and 12873 ideals in 1833 cliques
Mon Dec 24 13:41:01 2007 commencing in-memory singleton removal
Mon Dec 24 13:41:02 2007 begin with 5890502 relations and 4078637 unique ideals
Mon Dec 24 13:41:03 2007 reduce to 5890490 relations and 4065752 ideals in 4 passes
Mon Dec 24 13:41:03 2007 max relations containing the same ideal: 43
Mon Dec 24 13:41:04 2007 dataset has 70.0% excess relations
Mon Dec 24 13:41:08 2007 ignoring smallest 486785 rational and 486649 algebraic ideals
Mon Dec 24 13:41:08 2007 filtering ideals above 7159635
Mon Dec 24 13:41:08 2007 need 1642264 more relations than ideals
Mon Dec 24 13:41:08 2007 commencing singleton removal, final pass
Mon Dec 24 13:42:04 2007 memory use: 221.0 MB
Mon Dec 24 13:42:04 2007 commencing in-memory singleton removal
Mon Dec 24 13:42:05 2007 begin with 5890490 relations and 4165582 unique ideals
Mon Dec 24 13:42:07 2007 reduce to 5889251 relations and 4164343 ideals in 5 passes
Mon Dec 24 13:42:07 2007 max relations containing the same ideal: 46
Mon Dec 24 13:42:09 2007 removing 207355 relations and 166033 ideals in 41322 cliques
Mon Dec 24 13:42:09 2007 commencing in-memory singleton removal
Mon Dec 24 13:42:10 2007 begin with 5681896 relations and 4164343 unique ideals
Mon Dec 24 13:42:12 2007 reduce to 5677850 relations and 3994234 ideals in 5 passes
Mon Dec 24 13:42:12 2007 max relations containing the same ideal: 45
Mon Dec 24 13:42:14 2007 removing 182469 relations and 141147 ideals in 41322 cliques
Mon Dec 24 13:42:14 2007 commencing in-memory singleton removal
Mon Dec 24 13:42:15 2007 begin with 5495381 relations and 3994234 unique ideals
Mon Dec 24 13:42:17 2007 reduce to 5491480 relations and 3849163 ideals in 5 passes
Mon Dec 24 13:42:17 2007 max relations containing the same ideal: 43
Mon Dec 24 13:42:18 2007 dataset has 53.0% excess relations
Mon Dec 24 13:42:21 2007 ignoring smallest 436153 rational and 436291 algebraic ideals
Mon Dec 24 13:42:21 2007 filtering ideals above 6364120
Mon Dec 24 13:42:21 2007 need 1313853 more relations than ideals
Mon Dec 24 13:42:21 2007 commencing singleton removal, final pass
Mon Dec 24 13:43:14 2007 memory use: 176.9 MB
Mon Dec 24 13:43:14 2007 commencing in-memory singleton removal
Mon Dec 24 13:43:15 2007 begin with 5491480 relations and 3950115 unique ideals
Mon Dec 24 13:43:17 2007 reduce to 5491028 relations and 3949663 ideals in 5 passes
Mon Dec 24 13:43:17 2007 max relations containing the same ideal: 43
Mon Dec 24 13:43:19 2007 removing 437911 relations and 324155 ideals in 113756 cliques
Mon Dec 24 13:43:19 2007 commencing in-memory singleton removal
Mon Dec 24 13:43:20 2007 begin with 5053117 relations and 3949663 unique ideals
Mon Dec 24 13:43:22 2007 reduce to 5031002 relations and 3603031 ideals in 6 passes
Mon Dec 24 13:43:22 2007 max relations containing the same ideal: 41
Mon Dec 24 13:43:24 2007 removing 435009 relations and 321253 ideals in 113756 cliques
Mon Dec 24 13:43:24 2007 commencing in-memory singleton removal
Mon Dec 24 13:43:25 2007 begin with 4595993 relations and 3603031 unique ideals
Mon Dec 24 13:43:27 2007 reduce to 4571738 relations and 3257068 ideals in 6 passes
Mon Dec 24 13:43:27 2007 max relations containing the same ideal: 38
Mon Dec 24 13:43:29 2007 removing 6150 relations and 5334 ideals in 816 cliques
Mon Dec 24 13:43:29 2007 commencing in-memory singleton removal
Mon Dec 24 13:43:29 2007 begin with 4565588 relations and 3257068 unique ideals
Mon Dec 24 13:43:30 2007 reduce to 4565577 relations and 3251723 ideals in 3 passes
Mon Dec 24 13:43:30 2007 max relations containing the same ideal: 38
Mon Dec 24 13:43:31 2007 dataset has 22.4% excess relations
Mon Dec 24 13:43:33 2007 ignoring smallest 385176 rational and 385111 algebraic ideals
Mon Dec 24 13:43:33 2007 filtering ideals above 5568605
Mon Dec 24 13:43:33 2007 need 1159245 more relations than ideals
Mon Dec 24 13:43:33 2007 commencing singleton removal, final pass
Mon Dec 24 13:44:20 2007 memory use: 176.9 MB
Mon Dec 24 13:44:20 2007 commencing in-memory singleton removal
Mon Dec 24 13:44:21 2007 begin with 4565577 relations and 3353856 unique ideals
Mon Dec 24 13:44:22 2007 reduce to 4565253 relations and 3353532 ideals in 4 passes
Mon Dec 24 13:44:22 2007 max relations containing the same ideal: 38
Mon Dec 24 13:44:24 2007 removing 125103 relations and 98865 ideals in 26238 cliques
Mon Dec 24 13:44:24 2007 commencing in-memory singleton removal
Mon Dec 24 13:44:24 2007 begin with 4440150 relations and 3353532 unique ideals
Mon Dec 24 13:44:26 2007 reduce to 4438353 relations and 3252856 ideals in 6 passes
Mon Dec 24 13:44:26 2007 max relations containing the same ideal: 37
Mon Dec 24 13:44:28 2007 removing 109556 relations and 83318 ideals in 26238 cliques
Mon Dec 24 13:44:28 2007 commencing in-memory singleton removal
Mon Dec 24 13:44:29 2007 begin with 4328797 relations and 3252856 unique ideals
Mon Dec 24 13:44:30 2007 reduce to 4327258 relations and 3167992 ideals in 4 passes
Mon Dec 24 13:44:30 2007 max relations containing the same ideal: 36
Mon Dec 24 13:44:31 2007 dataset has 8.0% excess relations
Mon Dec 24 13:44:31 2007 relations with 0 large ideals: 64186
Mon Dec 24 13:44:31 2007 relations with 1 large ideals: 166820
Mon Dec 24 13:44:31 2007 relations with 2 large ideals: 788817
Mon Dec 24 13:44:31 2007 relations with 3 large ideals: 1452146
Mon Dec 24 13:44:31 2007 relations with 4 large ideals: 1258160
Mon Dec 24 13:44:31 2007 relations with 5 large ideals: 504940
Mon Dec 24 13:44:31 2007 relations with 6 large ideals: 72366
Mon Dec 24 13:44:31 2007 relations with 7+ large ideals: 19823
Mon Dec 24 13:44:31 2007 commencing 2-way merge
Mon Dec 24 13:44:34 2007 reduce to 3027563 relation sets and 1868297 unique ideals
Mon Dec 24 13:44:34 2007 commencing full merge
Mon Dec 24 13:45:12 2007 found 1494530 cycles, need 1116497
Mon Dec 24 13:45:12 2007 weight of 1116497 cycles is about 72815516 (65.22/cycle)
Mon Dec 24 13:45:12 2007 distribution of cycle lengths:
Mon Dec 24 13:45:12 2007 1 relations: 163084
Mon Dec 24 13:45:12 2007 2 relations: 115469
Mon Dec 24 13:45:12 2007 3 relations: 113624
Mon Dec 24 13:45:12 2007 4 relations: 111286
Mon Dec 24 13:45:12 2007 5 relations: 107308
Mon Dec 24 13:45:12 2007 6 relations: 101566
Mon Dec 24 13:45:12 2007 7 relations: 95713
Mon Dec 24 13:45:12 2007 8 relations: 88257
Mon Dec 24 13:45:12 2007 9 relations: 80601
Mon Dec 24 13:45:12 2007 10+ relations: 139589
Mon Dec 24 13:45:12 2007 heaviest cycle: 14 relations
Mon Dec 24 13:45:12 2007 commencing cycle optimization
Mon Dec 24 13:45:14 2007 start with 5908059 relations
Mon Dec 24 13:45:27 2007 pruned 136471 relations
Mon Dec 24 13:45:27 2007 distribution of cycle lengths:
Mon Dec 24 13:45:27 2007 1 relations: 163084
Mon Dec 24 13:45:27 2007 2 relations: 117587
Mon Dec 24 13:45:27 2007 3 relations: 118032
Mon Dec 24 13:45:27 2007 4 relations: 114046
Mon Dec 24 13:45:27 2007 5 relations: 111624
Mon Dec 24 13:45:27 2007 6 relations: 105109
Mon Dec 24 13:45:27 2007 7 relations: 99071
Mon Dec 24 13:45:27 2007 8 relations: 90724
Mon Dec 24 13:45:27 2007 9 relations: 80174
Mon Dec 24 13:45:27 2007 10+ relations: 117046
Mon Dec 24 13:45:27 2007 heaviest cycle: 13 relations
Mon Dec 24 13:45:28 2007
Mon Dec 24 13:45:28 2007 commencing linear algebra
Mon Dec 24 13:45:28 2007 read 1116497 cycles
Mon Dec 24 13:45:30 2007 cycles contain 3019691 unique relations
Mon Dec 24 13:46:01 2007 read 3019691 relations
Mon Dec 24 13:46:05 2007 using 32 quadratic characters above 134217618
Mon Dec 24 13:47:21 2007 read 1116497 cycles
Mon Dec 24 13:47:22 2007 matrix is 1115243 x 1116497 with weight 96072869 (avg 86.05/col)
Mon Dec 24 13:47:34 2007 filtering completed in 3 passes
Mon Dec 24 13:47:35 2007 matrix is 1104877 x 1105077 with weight 95282556 (avg 86.22/col)
Mon Dec 24 13:47:42 2007 read 1105077 cycles
Mon Dec 24 13:47:43 2007 matrix is 1104877 x 1105077 with weight 95282556 (avg 86.22/col)
Mon Dec 24 13:47:43 2007 saving the first 48 matrix rows for later
Mon Dec 24 13:47:44 2007 matrix is 1104829 x 1105077 with weight 72022606 (avg 65.17/col)
Mon Dec 24 13:47:44 2007 matrix includes 64 packed rows
Mon Dec 24 13:47:44 2007 using block size 65536 for processor cache size 4096 kB
Mon Dec 24 13:47:49 2007 commencing Lanczos iteration (4 threads)
Mon Dec 24 15:32:36 2007 lanczos halted after 23515 iterations (dim = 1487008)
[/code]

The 2-threaded run succeeded.

By the way, I'm talking to Brian and will try to work out what's going on here with him.