Newer X64 build needed

Googulator · 2015-12-30, 12:28

Hi,

The latest usable X64 binary on Jeff Gilchrist's page is SVN 883, and the latest one for Core 2 CPUs is 1.50. (There are SVN 939 and 942 binaries, but those are buggy and crash in the linear algebra.) On this forum, there exists a build from SVN 946, but it's built without ECM support.

1.52 is available on SourceForge, but only 32-bit. In my testing, 32-bit builds are a lot slower, especially the ECM.

I don't know if Msieve's SF maintainer reads this forum, but if he does, please make a 64-bit build of the official 1.52!

jasonp · 2015-12-30, 13:44

Msieve's maintainer does read this forum, and should release v1.53 since it's been languishing for a year. I also now have the capability to build x64 binaries so those should move to SF too.

The slower ECM does not surprise me, but in my experience latter-day GMP has a lot of difficulty building on 64-bit windows.

bsquared · 2015-12-30, 14:55

Quote:

Originally Posted by jasonp

The slower ECM does not surprise me, but in my experience latter-day GMP has a lot of difficulty building on 64-bit windows.

It requires an up-to-date mingw-64/msys toolchain. ATH seems to know quite a bit about how to make it work (e.g. http://www.mersenneforum.org/showthr...4087&page=36); maybe he'd be willing to take questions if something doesn't work for you. I don't know how this reconciles with the GPU portion of the msieve build... or if there is a way to get latter-day GMP to compile with microsoft/visual studio...

wombatman · 2015-12-30, 15:09

Yeah, GMP is not an issue with MinGW-64. I had a harder time building MSieve, especially in terms of the issues when using a CUDA version newer than 5.5.

ATH · 2015-12-30, 19:36

Actually I learned all about 64 bit compiling with Mingw64 from mainly WraithX but also from wombatman, so they know a lot more than me. I have never compiled a CUDA application but it was a long time since I tried.

ATH · 2015-12-30, 22:22

I installed CUDA 6.5 (since 7.0 and 7.5 does not work for mfaktc I assumed it was best to avoid them for msieve as well).

I added the CUDA path and various flags to the makefile but it fails when it reaches the nvcc step. Do I really need Visual Studio installed to compile a CUDA application when I'm trying to compile with Msys2+Mingw64? I have VS2015 but it needs 2010/2012/2013:

Code:

"/C/CUDA6.5/bin/nvcc" -arch sm_11 -ptx -o stage1_core_sm11.ptx gnfs/poly/stage1/   stage1_core_gpu/stage1_core.cu
nvcc warning : The 'compute_11', 'compute_12', 'compute_13', 'sm_11', 'sm_12', a   nd 'sm_13' architectures are deprecated, and may be removed in a future release.
nvcc fatal   : nvcc cannot find a supported version of Microsoft Visual Studio.    Only the versions 2010, 2012, and 2013 are supported
Makefile:307: recipe for target 'stage1_core_sm11.ptx' failed
make: *** [stage1_core_sm11.ptx] Error 1

jasonp · 2015-12-31, 00:05

My problem is not that GMP doesn't work, it's that Windows 7 has some kind of background service that holds onto executables just long enough for the configure script to fail trying to delete temporary exe's that it generates. Thus configure doesn't work and I can't get to the next step.

People all over the internet have the same problem, and I've tried all of the solutions they propose and nothing works.

I can compile a generic build of 64-bit MPIR using Visual Studio, but the last time I tried compiling the assembler code YASM had some kind of problem (late last year).

Getting something that can execute GMP function calls on 64-bit windows has been incredibly frustrating for me, it has exceeded the time limit I could devote to it every single time.

I could probably cross-compile a 64-bit build on a widows XP system, but the latest CUDA doesn't work with MSVC2008 and MSVC2010 doesn't work on XP. Hence the year-long stalemate where I can't update the CUDA code to work with CUDA > 5.5 either.

ATH · 2015-12-31, 01:57

Have you tried MSYS2 with Mingw64?:
https://sourceforge.net/p/msys2/wiki...0installation/

I ran these commands to update it initially:

Code:

Run "msys2_shell.bat"

update-core

Restart MSYS2 (using msys2_shell.bat)

pacman -Su

Restart MSYS2 (using mingw64_shell.bat and use this to start MSYS2 from now on)

pacman -S mingw-w64-x86_64-gcc
pacman -S mingw-w64-x86_64-make
pacman -S mingw-w64-x86_64-libtool
pacman -S autoconf
pacman -S automake
pacman -S make

If this does not work or if you just want to try it quick, I saved my installation after these steps above: msys64.zip
(It's a 7-zip file but the site does not allow .7z extention. Rename it to msys64.7z)
It is simply the msys64 folder which should be extracted to C: and then run mingw64_shell.bat.

I compile GMP 6.1.0 on a Haswell with:

Code:

./configure ABI=64 CC=gcc CFLAGS="-O3 -m64 -mavx -mavx2 -mfma -march=haswell -mtune=haswell" --build=haswell-w64-mingw32 --enable-static --disable-shared
make
make install
make check

It works fine with MSYS2 in both Windows 7 and 10.

Googulator · 2015-12-31, 18:52

I see you're using Haswell-specific optimizations. Please make also a generic build, or at least something usable on Core 2. (Mine has SSE4.1, but not all Core 2s have it.)

ATH · 2015-12-31, 22:29

I compiled the newest svn988 without CUDA support with haswell, sandy bridge and core2 flags. I got a fair number of warnings most are "unused-parameter" but there are other that might be serious?

I will not post any binaries until we check if the warnings are ok:

Code:

common/lanczos/lanczos.c: In function 'dump_lanczos_state':
common/lanczos/lanczos.c:485:21: warning: unused parameter 'packed_matrix' [-Wunused-parameter]
    packed_matrix_t *packed_matrix,
                     ^
common/lanczos/lanczos.c:488:11: warning: unused parameter 'n' [-Wunused-parameter]
    uint32 n, uint32 max_n, uint32 dim_solved, uint32 iter,
           ^
common/lanczos/lanczos.c: In function 'read_lanczos_state':
common/lanczos/lanczos.c:643:21: warning: unused parameter 'packed_matrix' [-Wunused-parameter]
    packed_matrix_t *packed_matrix,
                     ^
common/lanczos/lanczos.c:646:11: warning: unused parameter 'n' [-Wunused-parameter]
    uint32 n, uint32 max_n, uint32 *dim_solved,
           ^
common/lanczos/lanczos_io.c: In function 'dump_matrix':
common/lanczos/lanczos_io.c:173:10: warning: unused parameter 'sparse_weight' [-Wunused-parameter]
   uint64 sparse_weight) {
          ^
common/lanczos/lanczos_io.c: In function 'file_cache_get_next':
common/lanczos/lanczos_io.c:372:45: warning: unused parameter 'obj' [-Wunused-parameter]
 static void file_cache_get_next(msieve_obj *obj, FILE *fp,
                                             ^
common/lanczos/lanczos_io.c:375:12: warning: unused parameter 'read_submatrix' [-Wunused-parameter]
     uint32 read_submatrix) {
            ^
common/lanczos/lanczos_io.c: In function 'read_matrix':
common/lanczos/lanczos_io.c:438:23: warning: variable 'mpi_nrows' set but not used [-Wunused-but-set-variable]
  uint32 mpi_resclass, mpi_nrows;
                       ^
common/lanczos/lanczos_io.c:438:9: warning: variable 'mpi_resclass' set but not used [-Wunused-but-set-variable]
  uint32 mpi_resclass, mpi_nrows;
         ^
common/lanczos/lanczos_matmul0.c: In function 'packed_matrix_init':
common/lanczos/lanczos_matmul0.c:419:12: warning: unused variable 'j' [-Wunused-variable]
  uint32 i, j;
            ^
common/lanczos/lanczos_matmul0.c: In function 'mul_MxN_Nx64':
common/lanczos/lanczos_matmul0.c:619:23: warning: unused parameter 'scratch' [-Wunused-parameter]
    uint64 *b, uint64 *scratch) {
                       ^
common/lanczos/lanczos_matmul1.c: In function 'mul_packed_core':
common/lanczos/lanczos_matmul1.c:308:38: warning: unused parameter 'thread_num' [-Wunused-parameter]
 void mul_packed_core(void *data, int thread_num)
                                      ^
common/lanczos/lanczos_matmul1.c: In function 'mul_packed_small_core':
common/lanczos/lanczos_matmul1.c:348:44: warning: unused parameter 'thread_num' [-Wunused-parameter]
 void mul_packed_small_core(void *data, int thread_num)
                                            ^
common/lanczos/lanczos_matmul2.c: In function 'mul_trans_packed_core':
common/lanczos/lanczos_matmul2.c:319:44: warning: unused parameter 'thread_num' [-Wunused-parameter]
 void mul_trans_packed_core(void *data, int thread_num)
                                            ^
common/lanczos/lanczos_matmul2.c: In function 'mul_trans_packed_small_core':
common/lanczos/lanczos_matmul2.c:358:50: warning: unused parameter 'thread_num' [-Wunused-parameter]
 void mul_trans_packed_small_core(void *data, int thread_num)
                                                  ^
common/lanczos/lanczos_vv.c: In function 'mul_Nx64_64x64_acc':
common/lanczos/lanczos_vv.c:201:9: warning: unused variable 'i' [-Wunused-variable]
  uint32 i;
         ^
common/lanczos/lanczos_vv.c: In function 'outer_thread_run':
common/lanczos/lanczos_vv.c:210:46: warning: unused parameter 'thread_num' [-Wunused-parameter]
 static void outer_thread_run(void *data, int thread_num)
                                              ^
common/lanczos/lanczos_vv.c: In function 'inner_thread_run':
common/lanczos/lanczos_vv.c:427:46: warning: unused parameter 'thread_num' [-Wunused-parameter]
 static void inner_thread_run(void *data, int thread_num)
                                              ^
common/lanczos/lanczos_vv.c: In function 'tmul_64xN_Nx64':
common/lanczos/lanczos_vv.c:441:12: warning: unused variable 'j' [-Wunused-variable]
  uint32 i, j;
            ^
common/smallfact/smallfact.c: In function 'trial_factor':
common/smallfact/smallfact.c:22:9: warning: variable 'factor_found' set but not used [-Wunused-but-set-variable]
  uint32 factor_found = 0;
         ^
common/minimize.c: In function 'solve_dmatrix':
common/minimize.c:421:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < n; i++)
                ^
common/minimize.c:424:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < n - 1; i++) {
                ^
common/minimize.c:431:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (j = i + 1; j < n; j++) {
                     ^
common/minimize.c:444:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (j = i + 1; j < n; j++) {
                     ^
common/minimize.c:448:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (k = i + 1; k < n; k++) {
                      ^
common/minimize.c:460:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (j = i + 1; j < n; j++) {
                     ^
common/savefile.c: In function 'savefile_open':
common/savefile.c:131:9: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
   s->fp = gzopen(name_gz, open_string);
         ^
common/savefile.c:157:10: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
    s->fp = gzopen(s->name, "a");
          ^
common/savefile.c:163:9: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
   s->fp = gzopen(s->name, open_string);
         ^
common/savefile.c: In function 'savefile_close':
common/savefile.c:182:49: warning: passing argument 1 of 'gzclose' from incompatible pointer type [-Wincompatible-pointer-types]
  s->is_a_FILE ? fclose((FILE *)s->fp) : gzclose(s->fp);
                                                 ^
In file included from include/util.h:46:0,
                 from include/msieve.h:24,
                 from include/common.h:18,
                 from common/savefile.c:15:
C:/msys64/mingw64/include/zlib.h:1511:24: note: expected 'gzFile {aka struct gzFile_s *}' but argument is of type 'struct gzFile_s **'
 ZEXTERN int ZEXPORT    gzclose OF((gzFile file));
                        ^
common/savefile.c: In function 'savefile_eof':
common/savefile.c:193:53: warning: passing argument 1 of 'gzeof' from incompatible pointer type [-Wincompatible-pointer-types]
  return (s->is_a_FILE ? feof((FILE *)s->fp) : gzeof(s->fp));
                                                     ^
In file included from include/util.h:46:0,
                 from include/msieve.h:24,
                 from include/common.h:18,
                 from common/savefile.c:15:
C:/msys64/mingw64/include/zlib.h:1475:21: note: expected 'gzFile {aka struct gzFile_s *}' but argument is of type 'struct gzFile_s **'
 ZEXTERN int ZEXPORT gzeof OF((gzFile file));
                     ^
common/savefile.c: In function 'savefile_read_line':
common/savefile.c:251:9: warning: passing argument 1 of 'gzgets' from incompatible pointer type [-Wincompatible-pointer-types]
  gzgets(s->fp, buf, (int)max_len);
         ^
In file included from include/util.h:46:0,
                 from include/msieve.h:24,
                 from include/common.h:18,
                 from common/savefile.c:15:
C:/msys64/mingw64/include/zlib.h:1372:24: note: expected 'gzFile {aka struct gzFile_s *}' but argument is of type 'struct gzFile_s **'
 ZEXTERN char * ZEXPORT gzgets OF((gzFile file, char *buf, int len));
                        ^
common/savefile.c: In function 'savefile_flush':
common/savefile.c:279:10: warning: passing argument 1 of 'gzputs' from incompatible pointer type [-Wincompatible-pointer-types]
   gzputs(s->fp, s->buf);
          ^
In file included from include/util.h:46:0,
                 from include/msieve.h:24,
                 from include/common.h:18,
                 from common/savefile.c:15:
C:/msys64/mingw64/include/zlib.h:1364:21: note: expected 'gzFile {aka struct gzFile_s *}' but argument is of type 'struct gzFile_s **'
 ZEXTERN int ZEXPORT gzputs OF((gzFile file, const char *s));
                     ^
common/savefile.c: In function 'savefile_rewind':
common/savefile.c:298:50: warning: passing argument 1 of 'gzrewind' from incompatible pointer type [-Wincompatible-pointer-types]
  s->is_a_FILE ? rewind((FILE *)s->fp) : gzrewind(s->fp);
                                                  ^
In file included from include/util.h:46:0,
                 from include/msieve.h:24,
                 from include/common.h:18,
                 from common/savefile.c:15:
C:/msys64/mingw64/include/zlib.h:1447:24: note: expected 'gzFile {aka struct gzFile_s *}' but argument is of type 'struct gzFile_s **'
 ZEXTERN int ZEXPORT    gzrewind OF((gzFile file));
                        ^
common/util.c: In function 'aligned_malloc':
common/util.c:40:9: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
  addr = (unsigned long)ptr;
         ^
common/util.c: In function 'get_cpu_type':
common/util.c:456:17: warning: variable 'model' set but not used [-Wunused-but-set-variable]
   uint8 family, model;
                 ^
mpqs/relation.c: In function 'qs_filter_relations':
mpqs/relation.c:807:9: warning: variable 'poly_saved' set but not used [-Wunused-but-set-variable]
  uint32 poly_saved;
         ^
mpqs/relation.c:804:21: warning: variable 'curr_poly_idx' set but not used [-Wunused-but-set-variable]
  uint32 curr_a_idx, curr_poly_idx, curr_rel;
                     ^
gnfs/poly/poly.c: In function 'read_poly':
gnfs/poly/poly.c:28:8: warning: variable 'status' set but not used [-Wunused-but-set-variable]
  int32 status = 0;
        ^
gnfs/poly/poly_skew.c: In function 'sizeopt_callback':
gnfs/poly/poly_skew.c:85:37: warning: unused parameter 'deg' [-Wunused-parameter]
 static void sizeopt_callback(uint32 deg, mpz_t *alg_coeffs, mpz_t *rat_coeffs,
                                     ^
gnfs/poly/stage2/optimize_deg6.c: In function 'poly_eval':
gnfs/poly/stage2/optimize_deg6.c:208:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (k = 0; k < pow; k++)
                   ^
gnfs/poly/stage2/optimize_deg6.c: In function 'fill_powers':
gnfs/poly/stage2/optimize_deg6.c:406:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (j = 2; j <= max_pow; j++)
                 ^
gnfs/poly/stage2/optimize_deg6.c: In function 'optimize_initial_deg6':
gnfs/poly/stage2/optimize_deg6.c:722:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < (1 << (num_vars - 1)); i++) {
                ^
gnfs/ffpoly.c: In function 'get_zeros_rec':
gnfs/ffpoly.c:594:9: warning: 'g[0u].degree' may be used uninitialized in this function [-Wmaybe-uninitialized]
  poly_t g, xpow;
         ^
gnfs/relation.c:23:8: warning: always_inline function might not be inlinable [-Wattributes]
 uint32 divide_factor_out(mpz_t polyval, uint64 p,
        ^

ATH · 2015-12-31, 22:34

In the attempt to compile Msieve with CUDA support I installed Visual Studio 2013 and CUDA 6.5 on my laptop, and it got a bit further than it did with VS2015 on my desktop.

But it fails when reaching the "sm_10, compute_10" step, can this be disabled as this is deprecated I believe?

Code:

make[1]: Entering directory '/home/ATH/msieve/b40c'
"C:\CUDA6.5/bin/nvcc" -gencode=arch=compute_10,code=\"sm_10,compute_10\"  -o sort_engine_sm10.dll sort_engine.cu -Xptxas -v -Xcudafe -# -shared -Xptxas -abi=no -I"C:\CUDA6.5/include" -I.  -O3
nvcc fatal   : Unsupported gpu architecture 'compute_10'
Makefile:42: recipe for target 'sort_engine_sm10.dll' failed
make[1]: *** [sort_engine_sm10.dll] Error 1
make[1]: Leaving directory '/home/ATH/msieve/b40c'
Makefile:316: recipe for target 'b40c/built' failed
make: *** [b40c/built] Error 2

2015-12-30, 12:28	#1
Googulator Dec 2015 2²×5 Posts	Newer X64 build needed Hi, The latest usable X64 binary on Jeff Gilchrist's page is SVN 883, and the latest one for Core 2 CPUs is 1.50. (There are SVN 939 and 942 binaries, but those are buggy and crash in the linear algebra.) On this forum, there exists a build from SVN 946, but it's built without ECM support. 1.52 is available on SourceForge, but only 32-bit. In my testing, 32-bit builds are a lot slower, especially the ECM. I don't know if Msieve's SF maintainer reads this forum, but if he does, please make a 64-bit build of the official 1.52!

2015-12-31, 00:05	#7
jasonp Tribal Bullet Oct 2004 5×23×31 Posts	My problem is not that GMP doesn't work, it's that Windows 7 has some kind of background service that holds onto executables just long enough for the configure script to fail trying to delete temporary exe's that it generates. Thus configure doesn't work and I can't get to the next step. People all over the internet have the same problem, and I've tried all of the solutions they propose and nothing works. I can compile a generic build of 64-bit MPIR using Visual Studio, but the last time I tried compiling the assembler code YASM had some kind of problem (late last year). Getting something that can execute GMP function calls on 64-bit windows has been incredibly frustrating for me, it has exceeded the time limit I could devote to it every single time. I could probably cross-compile a 64-bit build on a widows XP system, but the latest CUDA doesn't work with MSVC2008 and MSVC2010 doesn't work on XP. Hence the year-long stalemate where I can't update the CUDA code to work with CUDA > 5.5 either. Last fiddled with by jasonp on 2015-12-31 at 00:07

2015-12-31, 01:57	#8
ATH Einyen Dec 2003 Denmark 3450₁₀ Posts	Have you tried MSYS2 with Mingw64?: https://sourceforge.net/p/msys2/wiki...0installation/ I ran these commands to update it initially: Code: Run "msys2_shell.bat" update-core Restart MSYS2 (using msys2_shell.bat) pacman -Su Restart MSYS2 (using mingw64_shell.bat and use this to start MSYS2 from now on) pacman -S mingw-w64-x86_64-gcc pacman -S mingw-w64-x86_64-make pacman -S mingw-w64-x86_64-libtool pacman -S autoconf pacman -S automake pacman -S make If this does not work or if you just want to try it quick, I saved my installation after these steps above: msys64.zip (It's a 7-zip file but the site does not allow .7z extention. Rename it to msys64.7z) It is simply the msys64 folder which should be extracted to C: and then run mingw64_shell.bat. I compile GMP 6.1.0 on a Haswell with: Code: ./configure ABI=64 CC=gcc CFLAGS="-O3 -m64 -mavx -mavx2 -mfma -march=haswell -mtune=haswell" --build=haswell-w64-mingw32 --enable-static --disable-shared make make install make check It works fine with MSYS2 in both Windows 7 and 10. Last fiddled with by ATH on 2015-12-31 at 01:58

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Newer milestone thread	Uncwilly	Data	3668	2023-04-20 12:24
Help needed - M77 232 917 celebration build	jbpace	Hardware	19	2018-01-10 16:32
Performance of cuda-ecm on newer hardware?	fivemack	GMP-ECM	14	2015-02-12 20:10
Newer msieves are slow on Core i7	mklasson	Msieve	9	2009-02-18 12:58
Use of large memory pages possible with newer linux kernels	Dresdenboy	Software	3	2003-12-08 14:47

2015-12-30, 13:44	#2
jasonp Tribal Bullet Oct 2004 DED₁₆ Posts	Msieve's maintainer does read this forum, and should release v1.53 since it's been languishing for a year. I also now have the capability to build x64 binaries so those should move to SF too. The slower ECM does not surprise me, but in my experience latter-day GMP has a lot of difficulty building on 64-bit windows.

2015-12-30, 15:09	#4
wombatman I moo ablest echo power! May 2013 2⁶·29 Posts	Yeah, GMP is not an issue with MinGW-64. I had a harder time building MSieve, especially in terms of the issues when using a CUDA version newer than 5.5.

2015-12-30, 19:36	#5
ATH Einyen Dec 2003 Denmark 2·3·5²·23 Posts	Actually I learned all about 64 bit compiling with Mingw64 from mainly WraithX but also from wombatman, so they know a lot more than me. I have never compiled a CUDA application but it was a long time since I tried.

2015-12-31, 18:52	#9
Googulator Dec 2015 2²×5 Posts	I see you're using Haswell-specific optimizations. Please make also a generic build, or at least something usable on Core 2. (Mine has SSE4.1, but not all Core 2s have it.)