mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Factoring

Reply
 
Thread Tools
Old 2021-02-10, 21:27   #925
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

277 Posts
Default

Quote:
Originally Posted by henryzz View Post
Is this using MPI? On a multi-cpu system that should help.
I don't have a lot of experience using MPI. I tried installing mpirun and the necessary developer libs, and recompiled msieve with MPI=1. I'm getting this error when trying to run, though:

Code:
$ mpirun -np 36 ~/math/msieve-mpi/msieve -v -i ./input.ini -l ./input.log  -s ./input.dat -nf ./input.fb -nc2 2,18
Invalid MIT-MAGIC-COOKIE-1 keyInvalid MIT-MAGIC-COOKIE-1 key[workstation:3437987] *** Process received signal ***
[workstation:3437987] Signal: Segmentation fault (11)
[workstation:3437987] Signal code: Invalid permissions (2)
[workstation:3437987] Failing at address: 0x558c64230990
[workstation:3437987] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x3bd60)[0x7fa355b7bd60]
ryanp is offline   Reply With Quote
Old 2021-02-11, 17:43   #926
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

10110111000112 Posts
Default

Quote:
Originally Posted by ryanp View Post
I don't have a lot of experience using MPI. I tried installing mpirun and the necessary developer libs, and recompiled msieve with MPI=1. I'm getting this error when trying to run, though:

Code:
$ mpirun -np 36 ~/math/msieve-mpi/msieve -v -i ./input.ini -l ./input.log  -s ./input.dat -nf ./input.fb -nc2 2,18
Invalid MIT-MAGIC-COOKIE-1 keyInvalid MIT-MAGIC-COOKIE-1 key[workstation:3437987] *** Process received signal ***
[workstation:3437987] Signal: Segmentation fault (11)
[workstation:3437987] Signal code: Invalid permissions (2)
[workstation:3437987] Failing at address: 0x558c64230990
[workstation:3437987] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x3bd60)[0x7fa355b7bd60]
I am no expert on MPI running although there supposed to be a speedup. Posts #229 and #230 from https://mersenneforum.org/showthread...=24211&page=13 would be worth reading.

After googling MIT-MAGIC-COOKIE-1 errors seem to be related to x11. Could you try in console mode? unsetting the display variable may be enough "unset DISPLAY"

There have been issues with different versions of mpi/ubuntu. EdH might be able to help sort through that mess. This could be the same issue.
henryzz is offline   Reply With Quote
Old 2021-02-13, 15:16   #927
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

11516 Posts
Default

I'm back to sieving for this job again. Currently at 2,136,576,912 uniques. My plan for each ~day: gather new relations sieved, then do an msieve filtering run to see how large the resultant matrix is.
ryanp is offline   Reply With Quote
Old 2021-02-17, 20:37   #928
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

277 Posts
Default

Currently at:

Code:
found 0 duplicates and 2472407741 unique relations
memory use: 31280.0 MB
reading ideals above 2873819136
commencing singleton removal, initial pass
memory use: 41024.0 MB
reading all ideals from disk
memory use: 41399.9 MB
commencing in-memory singleton removal
begin with 2472407741 relations and 2048964861 unique ideals
reduce to 1444557329 relations and 911198417 ideals in 16 passes
max relations containing the same ideal: 34
reading ideals above 720000
commencing singleton removal, initial pass
With ~2.28B uniques, the matrix size was down to 134M, so hopefully we can push it down a bit further still.
ryanp is offline   Reply With Quote
Old 2021-02-17, 23:45   #929
RichD
 
RichD's Avatar
 
Sep 2008
Kansas

2×3×7×79 Posts
Default

No wonder this has been the number one most wanted for 15 years or so...
RichD is offline   Reply With Quote
Old 2021-02-19, 22:06   #930
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

277 Posts
Default

Currently running the filtering with 2.59B uniques, and target_density=140.

Previous run (2.47B uniques) managed to get the matrix size down to 125M.
ryanp is offline   Reply With Quote
Old 2021-02-19, 23:33   #931
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

16E316 Posts
Default

What's the target? How does the solving time scale?
henryzz is offline   Reply With Quote
Old 2021-02-20, 14:46   #932
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

1000101012 Posts
Default

Quote:
Originally Posted by henryzz View Post
What's the target? How does the solving time scale?
Not sure about the target; something in the 90M to 100M range would be nice. I'm also running up against the sheer size of the relations file, which is now about 200GB even gzip'ed.
ryanp is offline   Reply With Quote
Old 2021-02-21, 18:38   #933
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

27710 Posts
Default

Assuming the current filtering run (2.59B uniques, target_density=140) succeeds, I will probably start LA afterward. The size of the relations file is becoming the issue at this point.
ryanp is offline   Reply With Quote
Old 2021-02-22, 14:52   #934
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

277 Posts
Default

Code:
commencing linear algebra
read 118001756 cycles
cycles contain 408993293 unique relations
read 408993293 relations
using 20 quadratic characters above 4294917295
building initial matrix
memory use: 54264.7 MB
read 118001756 cycles
matrix is 118001577 x 118001756 (63095.7 MB) with weight 17614526827 (149.27/col)
sparse part has weight 15360150213 (130.17/col)
filtering completed in 2 passes
matrix is 117998763 x 117998931 (63095.5 MB) with weight 17614443699 (149.28/col)
sparse part has weight 15360119182 (130.17/col)
matrix starts at (0, 0)
matrix is 117998763 x 117998931 (63095.5 MB) with weight 17614443699 (149.28/col)
sparse part has weight 15360119182 (130.17/col)
saving the first 240 matrix rows for later
matrix includes 256 packed rows
matrix is 117998523 x 117998931 (58680.1 MB) with weight 14077102445 (119.30/col)
sparse part has weight 13494642461 (114.36/col)
using block size 8192 and superblock size 608256 for processor cache size 25344 kB
commencing Lanczos iteration (64 threads)
memory use: 80685.9 MB
linear algebra at 0.0%, ETA 2503h12m998931 dimensions (0.0%, ETA 2503h12m)
checkpointing every 50000 dimensions998931 dimensions (0.0%, ETA 2521h18m)
linear algebra completed 484175 of 117998931 dimensions (0.4%, ETA 2391h22m)
Doable, but slow; fortunately frmky has offered to lend a hand with the LA in a few weeks' time.
ryanp is offline   Reply With Quote
Old 2021-03-05, 19:08   #935
ryanp
 
ryanp's Avatar
 
Jun 2012
Boulder, CO

1000101012 Posts
Default

After getting up to 2.8B uniques, filtering with target_density=150 produced a 109Mx109M matrix.

Code:
commencing linear algebra
read 109443915 cycles
cycles contain 382966629 unique relations
read 382966629 relations
using 20 quadratic characters above 4294917295
building initial matrix
memory use: 51260.2 MB
read 109443915 cycles
matrix is 109443738 x 109443915 (61867.3 MB) with weight 17248536158 (157.60/col)
sparse part has weight 15123709456 (138.19/col)
filtering completed in 2 passes
matrix is 109441975 x 109442140 (61867.2 MB) with weight 17248479767 (157.60/col)
sparse part has weight 15123686159 (138.19/col)
matrix starts at (0, 0)
matrix is 109441975 x 109442140 (61867.2 MB) with weight 17248479767 (157.60/col)
sparse part has weight 15123686159 (138.19/col)
saving the first 240 matrix rows for later
matrix includes 256 packed rows
matrix is 109441735 x 109442140 (57387.5 MB) with weight 13866272003 (126.70/col)
sparse part has weight 13292709138 (121.46/col)
using block size 8192 and superblock size 608256 for processor cache size 25344 kB
commencing Lanczos iteration (64 threads)
memory use: 77410.7 MB
linear algebra at 0.0%, ETA 2362h54m442140 dimensions (0.0%, ETA 2362h54m)    
checkpointing every 50000 dimensions442140 dimensions (0.0%, ETA 2353h21m)    
linear algebra completed 12252 of 109442140 dimensions (0.0%, ETA 2299h53m)
This would probably be doable for me, but will be faster on frmky's MPI cluster.
ryanp is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Passive Pascal Xyzzy GPU Computing 1 2017-05-17 20:22
Tesla P100 — 5.4 DP TeraFLOPS — Pascal Mark Rose GPU Computing 52 2016-07-02 12:11
Nvidia Pascal, a third of DP firejuggler GPU Computing 12 2016-02-23 06:55
Calculating perfect numbers in Pascal Elhueno Homework Help 5 2008-06-12 16:37
Factorization attempt to a c163 - a new Odd Perfect Number roadblock jchein1 Factoring 30 2005-05-30 14:43

All times are UTC. The time now is 14:49.

Thu Apr 22 14:49:57 UTC 2021 up 14 days, 9:30, 0 users, load averages: 1.66, 2.12, 2.33

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.