FastECPP software and >50000 digit primality proof (reposted from NMBRTHRY)

[Batalov]

Excellent. I'll write to Markus - he could tweak his intake script.

I will also suggest him to automate the situation that someone would submit Pari-formatted cert, as well.
Pari-formatted cert is an array of arrays, a naked object. Markus could do the conversion on his server side.

[xilman]

Originally Posted by Batalov

I built the ecpp and ecpp-mpi ver.0.4.0 per instructions. Everything works like a clock, - Andreas is a pro.

And so far, - it is blazingly fast. And it is a command-line tool.

Agreed. I tested both versions on R1031 with instant success.

My only complaint is that it seems the number to be tested must be given on the command line. It would be nice if it could be fed in via stdin.

Yes, I know this is a trivial change to the source, which I will make and feed back to the devs.

[xilman]

Originally Posted by xilman

Agreed. I tested both versions on R1031 with instant success.

My only complaint is that it seems the number to be tested must be given on the command line. It would be nice if it could be fed in via stdin.

Yes, I know this is a trivial change to the source, which I will make and feed back to the devs.

Started it runnng anyway. I want to see how long it takes on my system. 12 ecmnet clients are running simultaneously at nice 19; they appear to be letting ecpp-mpi have arounbd 85% of the cpu. The 6 processes are taking about 8G RAM - 5 workers at 1.5G and one master process of 0.5G

Progress so far:

MPI with 5 workers initialised, of which 5 are local.
GMP: include 6.2.1, lib 6.2.1
MPFR: include 4.1.0, lib 4.1.0
MPC: include 1.2.1, lib 1.2.1
MPFRCX: include 0.6.3, lib 0.6.3
PARI: include 2.13.3, lib 2.13.3
Could not open file 'cert1033.cert1' for reading.
Writing to 'cert1033.cert1'.
-- Time for class numbers up to Dmax=283248900: 696.9 (159.6)
  ***   Warning: increasing stack size to 536870912.
  ***   Warning: increasing stack size to 536870912.
  ***   Warning: increasing stack size to 536870912.
  ***   Warning: increasing stack size to 536870912.
  ***   Warning: increasing stack size to 536870912.
  ***   Warning: increasing stack size to 1073741824.
  ***   Warning: increasing stack size to 1073741824.
  ***   Warning: increasing stack size to 1073741824.
  ***   Warning: increasing stack size to 1073741824.
  ***   Warning: increasing stack size to 1073741824.
  ***   Warning: increasing stack size to 2147483648.
  ***   Warning: increasing stack size to 2147483648.
  ***   Warning: increasing stack size to 2147483648.
  ***   Warning: increasing stack size to 2147483648.
  ***   Warning: increasing stack size to 2147483648.
-- Time for primorial of B=2684354560: 269.2 (63.4)
-- hmaxprime: 32
-- Size [0]: 33660 bits

[paulunderwood]

Originally Posted by xilman

Agreed. I tested both versions on R1031 with instant success.

Even I managed to get mpirun ecpp-mpi -g -n '(10^1031-1)/9' -c -f cert-1031 working, after downloading, compiling and installing mpfrcx-0.6.3; and configuring ecpp-mpi with the right flag to configure; and installing openmpi and setting PATH and LD_LIBRARY_PATH -- all on a Celeron laptop.

As soon as my current Primo run has finished I will prove a Mersenne cofactor ~26k digits. It will be interesting to see how much saturation occurs on my 64 core machine.

[Stargate38]

I got every package to compile in Windows Bash, but when I run ecpp, I get "ecpp: error while loading shared libraries: libcm.so.0: cannot open shared object file: No such file or directory", and when I try to access the specified file via Windows, I get "The file cannot be accessed by the system". Googling the problem didn't help too much, but I did learn that the "file" (/usr/local/lib/libcm.so.0) in question is actually a broken symlink. How do I fix it?

[paulunderwood]

Originally Posted by Stargate38

I got every package to compile in Windows Bash, but when I run ecpp, I get "ecpp: error while loading shared libraries: libcm.so.0: cannot open shared object file: No such file or directory", and when I try to access the specified file via Windows, I get "The file cannot be accessed by the system". Googling the problem didn't help too much, but I did learn that the "file" (/usr/local/lib/libcm.so.0) in question is actually a broken symlink. How do I fix it?

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH

HTH.

Note ecpp, as opposed to ecpp-mpi, is single threaded. Your next step will be to get mpi running.

[frmky]

Certification of a 10k digit prime, 2^33217+111, took 20 hours using 2x Xeon E5-2670 v2 cpus with a total of 20 cores and 40 hyperthreads.

[Stargate38]

This is what happened when I tried running ecpp-mpi via "mpirun ecpp-mpi -g -n '10^1000+453' -c -f cert-1000" (The example provided in the accompanying PDF from their website):

-----------------------------------------------------------------------------
It is a Very Bad Idea to run this program as root.

LAM was designed to be run by individual users; it was *not* designed
to be run as a root-level service where multiple users use the same
LAM daemons in a client-server fashion.

Especially with today's propensity for hackers to scan for root-owned
network daemons, it could be tragic to run this program as root.
While LAM is known to be quite stable, and LAM does not leave network
sockets open for random connections after the initial setup, several
factors should strike fear into system administrator's hearts if LAM
were to be constantly running for all users to utilize:

        1. LAM leaves a Unix domain socket open on each machine in the
           /tmp directory.  So if someone breaks into root on one
           machine, they effectively have root on all machines that
           are connected via LAM.

        2. Indeed, there must have been a .rhosts (or some other trust
           mechanism) for root which must have allowed you to run LAM
           on remote nodes.  Depending on your local setup, this may
           not be safe.

        3. LAM has never been checked for buffer overflows and other
           malicious input types of errors.  We don't *think* that
           there are any buffer-overflow types of situations in LAM,
           we've never checked explicitly (hence, per Mr. Murphy,
           there are certainly some hiding somewhere).

        4. LAM programs are not audited or tracked in any way.  This
           could present a sneaky way to execute binaries without log
           trails (especially as root).

Hence, it's a Very Bad Idea to run LAM as root.  Please login as a
different user and run LAM again.
-----------------------------------------------------------------------------

Is there any way to bypass that? Root is the only user that I use in Windows Bash.

[paulunderwood]

Originally Posted by Stargate38

Is there any way to bypass that? Root is the only user that I use in Windows Bash.

Can you run?:

adduser stargate38
su stargate38

[xilman]

Originally Posted by paulunderwood

Can you run?:

adduser stargate38
su stargate38

+1

[xilman]

Originally Posted by paulunderwood

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH

HTH.

Note ecpp, as opposed to ecpp-mpi, is single threaded. Your next step will be to get mpi running.

The step after that is to learn how to drive mpi properly.

I am running mpi-ecpp on a 6-core machine. Without additional information it fired up one master process and 5 workers. Because of hyperthreading two processes could be interleaved at relatively little cost on each physical processor.

That said, each process will grab its own share of memory physical and virtual and you may not be able to run as many processes as processors, let alone twice that.

I a'm not going to fiddle with a(n apparently) working invocation of mpi-ecpp but now wonder whether "mpi -np 7" might be a suitable incantation for a similar job, on the grounds that there may (just) be enough RAM available.

The above assumes that the remaining machines on the local net are not set up to run mpi jobs. This is a valid assumption right now. Whether it is worth the effort of setting them up remains to be seen.