Mlucas on ubuntu

[Damian]

The test of M6972593 on my slow-as-tortoise x86 32-bit pc has just finished:

It was run non-stop from Nov 08 15:54:13, to Nov 10 01:11:57, so it lasted 1 day, 9 hours, 17 minutes and 44 seconds.

GP2, your estimate of 33 hours was pretty accurate!

This were the lasts lines written on p6972593.stat:

[Nov 10 00:46:57] M6972593 Iter# = 6900000 clocks = 00:04:04.188 [  0.0244 sec/iter] Res64: 14C5B1FD42863A75. AvgMaxErr = 0.160297802. MaxErr = 0.238441467
[Nov 10 00:51:22] M6972593 Iter# = 6910000 clocks = 00:04:24.592 [  0.0265 sec/iter] Res64: 3EC8150E85413CB3. AvgMaxErr = 0.160289397. MaxErr = 0.259231567
[Nov 10 00:55:16] M6972593 Iter# = 6920000 clocks = 00:03:54.132 [  0.0234 sec/iter] Res64: 7FD53969EAD17902. AvgMaxErr = 0.160376644. MaxErr = 0.269927979
[Nov 10 00:58:39] M6972593 Iter# = 6930000 clocks = 00:03:23.194 [  0.0203 sec/iter] Res64: 292C9075C71B15BF. AvgMaxErr = 0.160524616. MaxErr = 0.246856689
[Nov 10 01:01:55] M6972593 Iter# = 6940000 clocks = 00:03:15.463 [  0.0195 sec/iter] Res64: 2DF5D12473665FA8. AvgMaxErr = 0.160290362. MaxErr = 0.253234863
[Nov 10 01:05:10] M6972593 Iter# = 6950000 clocks = 00:03:15.594 [  0.0196 sec/iter] Res64: 39AC29E66DCAF8A6. AvgMaxErr = 0.160277283. MaxErr = 0.240447998
[Nov 10 01:08:22] M6972593 Iter# = 6960000 clocks = 00:03:11.969 [  0.0192 sec/iter] Res64: 147FCABF3BA439A8. AvgMaxErr = 0.160557336. MaxErr = 0.234924316
[Nov 10 01:11:13] M6972593 Iter# = 6970000 clocks = 00:02:50.746 [  0.0171 sec/iter] Res64: 037477199FF10F91. AvgMaxErr = 0.160435169. MaxErr = 0.244224548
[Nov 10 01:11:57] M6972593 Iter# = 6972591 clocks = 00:00:43.993 [  0.0170 sec/iter] Res64: 0000000000000000. AvgMaxErr = 0.041570618. MaxErr = 0.245819092
M6972593 is a known MERSENNE PRIME.

At the instant the test finished, the line of the exponent was removed from worktodo.ini file as expected, and this strange message was written in results.txt:

ERROR: Unrecognized/Unsupported option. The ini file entry was

Also, the mlucas job exited with this message on the console:

NTHREADS = 4
 looking for worktodo.ini file...
 worktodo.ini file found...checking next exponent in range...
ERROR: Unrecognized/Unsupported option. The ini file entry was 

ERROR: at line 547 of file ./src/Mlucas.c
Assertion failed:

I was expecting the p6972593 file to be written again with the last residue when the test finished, but that didn't happen. If I understand correctly, should that happened, the "N bytes for the bytewise residue R" should all be 0, right?

I don't know if the p6972593 format is standard (equal to prime95), but I think I would had written the checksums before the full residue, so all the important information would be at the begining of the file, and not both at the begining and ending. In the last hexdump I only was showing the ending, that was why the 0101 was not shown, but I looked and it is there indeed.

On the other hand, I succesfully compiled the Mlucas sourcecode in a newer notebook with AVX. I noticed the primenet.py script downloaded two DD tasks in the worktodo.ini, but when I log to primenet, these exponents show as Manual testing, that is, there seems not to be a way to assign a computer name and check the percentage done. Is that because primenet doesn't want other sofware rather than prime95 for that task, or just because the primenet.py script is in a early version and that could be added in a future?

And ewmayer, you are welcomed for my support. It's the first time I use paypal so I'm not sure how much time does it take, I still not see the transaction in the credit card in my bank homebanking. Hope it arrives, should be enough for a beer

[ewmayer]

Hi, Damian:

Originally Posted by Damian

At the instant the test finished, the line of the exponent was removed from worktodo.ini file as expected, and this strange message was written in results.txt:

ERROR: Unrecognized/Unsupported option. The ini file entry was

Also, the mlucas job exited with this message on the console:

NTHREADS = 4
 looking for worktodo.ini file...
 worktodo.ini file found...checking next exponent in range...
ERROR: Unrecognized/Unsupported option. The ini file entry was 

ERROR: at line 547 of file ./src/Mlucas.c
Assertion failed:

Not sure what might have gone awry there - I initially thought that might have been due to your having created a worktodo.ini file containing just the exponent and missing a trailing newline, but I just simulated under gdb with one of my own last-savefiles-prior-to-LL-test-completion, and stepping through the program logic after the final residues are written to the .stat file shows no issues. Without a case having the data needed (original ini-file and final p6972593 file) needed to reproduce the problem, not much I can do in terms of debug.

I was expecting the p6972593 file to be written again with the last residue when the test finished, but that didn't happen. If I understand correctly, should that happened, the "N bytes for the bytewise residue R" should all be 0, right?

No, the program only keeps the p-savefile from the final 10000-iteration (or 100000-iteration if more than 4 threads are used) checkpoint, it does not update it with the final LL residue. The reason is twofold:

[1] It is pretty trivial to rerun the final few thousand iterations, if needed for some reason;

[2] To prevent cheating. It is not hard to figure out the format of the savefile (the code used to create it is public, after all), and anyone mildly tech-savvy could use that format to create a valid-looking 0-final-residue savefile, since for res = 0 the residue checksums are all also zero. The nonzero residue a few hundred or more iterations prior to a final 0 residue, OTOH, is, so far as anyone knows, impossible to fake. So when a user's computer reports a new prime, the first step is to contact the user and ask for copies of the exponent logfile and the final checkpoint file, so the final few thousand (or however many) iterations can be independently run to see if the result truly is 0.

I don't know if the p6972593 format is standard (equal to prime95), but I think I would had written the checksums before the full residue, so all the important information would be at the begining of the file, and not both at the begining and ending. In the last hexdump I only was showing the ending, that was why the 0101 was not shown, but I looked and it is there indeed.

There is no GIMPS-standard format here, every program uses its own. Other users have previously suggested a common format so user A could hand off a savefile to user B to finish even if A and B are using different programs, but in fact there are good reasons why such mixing of clients on a single LL test is undesirable.

On the other hand, I succesfully compiled the Mlucas sourcecode in a newer notebook with AVX. I noticed the primenet.py script downloaded two DD tasks in the worktodo.ini, but when I log to primenet, these exponents show as Manual testing, that is, there seems not to be a way to assign a computer name and check the percentage done. Is that because primenet doesn't want other sofware rather than prime95 for that task, or just because the primenet.py script is in a early version and that could be added in a future?

Getting truly integrated primenet-comms is

[a] a big task, involving bringing in mutliple 3rd-party libraries. Not all of us are keen to introduce those code dependencies into our releases and build setups;

[b] breaks open-sourceness, since it requires a secret precompiled security module to be included.

OTOH, enhancing the server manual tests support to allow e.g. current-progress updates and individual machine IDs, that is worth discussing, but would depend on the current priment admin, Aaron Blosser (Madpoo around here) to be willing to undertake the upgrades. Feel free to post in the Primenet subforum if you wish to make the suggestion.

And ewmayer, you are welcomed for my support. It's the first time I use paypal so I'm not sure how much time does it take, I still not see the transaction in the credit card in my bank homebanking. Hope it arrives, should be enough for a beer

My p.s.-thanks was to let you know that I did receive your donation - it pretty much covers a whole case of beer (Lagunitas Brewing Co. IPA, in case anyone wants to know) from Costco, so 24 times thanks. :)

[Dubslow]

Originally Posted by ewmayer

[b] breaks open-sourceness, since it requires a secret precompiled security module to be included.

Not strictly true. The API is publicly available for use by any person and any program/client.

It is true that Prime95 includes secret code to aid with the integrity of data, and that the API supports transmission of said secret data, and that PrimeNet treats results without that secret integrity data with somewhat less trust than with.

However, the secret integrity data is *not* a requirement to use the API and the attendant benefits of the PrimeNet assignments interface. It merely results in the less-trusted handling of the result by the DB.

Damian, if you're so inclined, you could write a Python script (or other kind of script) that interfaces with the API. Or someone could even work it into the Mlucas code base, if they were so inclined. chalsall *does* have such scripts, but are written in the illegible Perl, and are also heavily integrated with the GPUto72 project and website for which those scripts were written.

[Damian]

I tried to run mlucas on a third machine. This time it is a notebook, the procesor has AVX, and the system installed is Debian jessie 32-bit.

After running

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

I search mlucas in the repository

sudo apt-cache search mlucas

but it returns nothing.

So I downloaded the source. When I compile with

gcc -c -O3 -DUSE_AVX -DUSE_THREADS ../src/*.c >& build.log

Then

grep -i error build.log

returns this kind of things

  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/radix16_dif_dit_pass.c:1114:124: error: macro "SSE2_RADIX16_DIF_TWIDDLE" passed 22 arguments, but takes just 11
../src/radix16_dif_dit_pass.c:1114:3: error: ‘SSE2_RADIX16_DIF_TWIDDLE’ undeclared (first use in this function)
../src/radix16_dif_dit_pass.c:2533:132: error: macro "SSE2_RADIX16_DIT_TWIDDLE" passed 24 arguments, but takes just 10
../src/radix16_dif_dit_pass.c:2533:3: error: ‘SSE2_RADIX16_DIT_TWIDDLE’ undeclared (first use in this function)
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!
../src/platform.h:1503:3: error: #error SIMD+AVX code only available for 64-bit build!
  #error  SIMD+AVX code only available for 64-bit build!

And when I compile with

gcc -c -O3 -DUSE_SSE2 -DUSE_THREADS ../src/*.c >& build.log

the grep error command

grep -i error build.log

returns again this kind of things:

../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm0’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm7’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm6’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm5’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm4’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm3’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm2’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm0’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm7’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm6’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm5’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm4’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm3’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm2’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm0’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm7’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm6’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm5’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm4’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm3’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm2’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/sse2_macro_gcc32.h:280:2: error: unknown register name ‘xmm0’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm6’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm5’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm3’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm2’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/twopmodq80.c:1283:2: error: unknown register name ‘xmm0’ in ‘asm’
../src/twopmodq80.c:1409:2: error: unknown register name ‘xmm6’ in ‘asm’
../src/twopmodq80.c:1409:2: error: unknown register name ‘xmm2’ in ‘asm’
../src/twopmodq80.c:1409:2: error: unknown register name ‘xmm1’ in ‘asm’
../src/twopmodq80.c:1409:2: error: unknown register name ‘xmm0’ in ‘asm’

I should have installed 64-bit debian or ubuntu *sigh*

So when a user's computer reports a new prime, the first step is to contact the user and ask for copies of the exponent logfile and the final checkpoint file, so the final few thousand (or however many) iterations can be independently run to see if the result truly is 0.

They *ask* copies of the logfile and final checkpoint file? If the email to them is automatic, why wouldn't they attach to that email that information? What if the user doesn't read the request, or doesn't answer for whatever reason?

Dubslow, thanks for the primenet api info. I might try my luck at a python script on next holidays in January.

[Xyzzy]

Originally Posted by Damian

I tried to run mlucas on a third machine. This time it is a notebook, the procesor has AVX, and the system installed is Debian jessie 32-bit.

After running

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

I search mlucas in the repository

sudo apt-cache search mlucas

but it returns nothing.

https://packages.debian.org/search?k...ll&section=all

It appears in "stretch", "buster" and "sid". (You could upgrade to "stretch" easily.)

[ewmayer]

Damian, it appears your gcc/binutils setup is broken w.r.to 32-bit builds. Just out of curiosity, what does 'gcc -v' return?

And, is there some particular reason for you to need a 32-bit OS? As your avx build tries show, beyond basic sse2 Mlucas requires a 64-bit platform - writing halfway-decent FFT code is hard enough with the 16 GPRs and 16 vector registers available in 64-bit mode, in 32-bit mode you lose access to half of those. IOW, running in 32-bit mode you are crippling much of your CPU's hardware capabilities.

[Damian]

Originally Posted by ewmayer

Damian, it appears your gcc/binutils setup is broken w.r.to 32-bit builds. Just out of curiosity, what does 'gcc -v' return?

And, is there some particular reason for you to need a 32-bit OS? As your avx build tries show, beyond basic sse2 Mlucas requires a 64-bit platform - writing halfway-decent FFT code is hard enough with the 16 GPRs and 16 vector registers available in 64-bit mode, in 32-bit mode you lose access to half of those. IOW, running in 32-bit mode you are crippling much of your CPU's hardware capabilities.

The strange thing is I tried on two different computers with different linux distribution and the same error showed up.

This is the output of gcc -v in the 32-bit PC with ubuntu

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-linux-gnu/5/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.5' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-i386/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-i386 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-i386 --with-arch-directory=i386 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-targets=all --enable-multiarch --disable-werror --with-arch-32=i686 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5)

And this is the output in the 64-bit notebook with debian 32-bit

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i586-linux-gnu/4.9/lto-wrapper
Target: i586-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.9.2-10' --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-i386/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-i386 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-i386 --with-arch-directory=i386 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-targets=all --enable-multiarch --with-arch-32=i586 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=i586-linux-gnu --host=i586-linux-gnu --target=i586-linux-gnu
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10)

Respect to why I instaled debian 32-bit in a 64-bit notebook, it was a stupid mistake I made, and plan to fix this summer. I guess I wasn't sure if all the programs I use would be compatible in 64bits as I always had used 32-bit until the last two or three years, but I guess I should have tried.