mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Linux (https://www.mersenneforum.org/forumdisplay.php?f=39)
-   -   Compile Linux version for PFGW (https://www.mersenneforum.org/showthread.php?t=22018)

carpetpool 2017-02-08 09:25

Compile Linux version for PFGW
 
The Windows version, it is the newest version of [URL="https://sourceforge.net/projects/openpfgw/files/?source=navbar"]PFGW[/URL]. The Linux version is outdated by half a year. Is someone able to compile a Linux version of the newest version of PFGW which is 3.8.3? Thanks to someone who is advanced at Linux and can do it, I am just a beginner at it for now. I can spare CPUs on my upgraded Linux computer as I cannot for Windows. The upgrade to PFGW is listed in [URL="http://www.mersenneforum.org/showthread.php?t=21779"]this[/URL] thread. :smile:

nordi 2017-02-08 21:04

I gave it a try. First, the source code needs a bit of adjustment to compile:


[CODE]--- pform/pfio/pfdecimalfile.cpp (revision 642)
+++ pform/pfio/pfdecimalfile.cpp (working copy)
@@ -2,6 +2,7 @@
#include "pfiopch.h"
#include <stdio.h>
#include <string.h>
+#include <ctype.h>

// also includes pffile.h but defines PFNewPGenFile class which is needed for the openInputFile() function.
#include "pfnewpgenfile.h"
[/CODE]Without that, it complains about isdigit() not being defined.

But then, "make" still fails with

[CODE]g++ -O3 -m64 -DX86_64 -D_64BIT -I../../packages/gmp/64bit -I../../pfconfig/headers \
pform/pfgw/.libs/pfgw_main.a pform/pfio/.libs/pfio.a pform/pfoo/.libs/pfoo.a pform/pfgwlib/.libs/pfgwlib.a \
pform/pfmath/.libs/pfmath.a pform/pflib/.libs/pflib.a \
packages/gmp/64bit/libgmp.a packages/gwnum/64bit/gwnum.a -Wl,-no_pie -lpthread -lstdc++ -o pfgw64
g++: error: packages/gmp/64bit/libgmp.a: No such file or directory
g++: error: packages/gwnum/64bit/gwnum.a: No such file or directory
makefile:8: recipe for target 'pfgw64' failed[/CODE]
These packages are not fetched/built by the makefile.

rogue 2017-02-08 21:26

You need to build gwnum and gmp on your own and move the libraries to the correct folders in order for it to build. Fortunately that should be fairly easy to do.

If you can point me to the correct source file for isdigit(), I can add it to the source. I have a couple of pending changes that I haven't committed yet so I will include it with those.

paulunderwood 2017-02-08 21:40

After installing gwnum.a and libgmp.a, and adding "#include <ctype.h>", my make gets me to:

[CODE]g++ -O3 -m64 -DX86_64 -D_64BIT -I../../packages/gmp/64bit -I../../pfconfig/headers \
pform/pfgw/.libs/pfgw_main.a pform/pfio/.libs/pfio.a pform/pfoo/.libs/pfoo.a pform/pfgwlib/.libs/pfgwlib.a \
pform/pfmath/.libs/pfmath.a pform/pflib/.libs/pflib.a \
packages/gmp/64bit/libgmp.a packages/gwnum/64bit/gwnum.a -Wl,-no_pie -lpthread -lstdc++ -o pfgw64
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
collect2: error: ld returned 1 exit status
makefile:8: recipe for target 'pfgw64' failed
make: *** [pfgw64] Error 1
[/CODE] :down:

nordi 2017-02-08 22:03

[QUOTE=rogue;452554]If you can point me to the correct source file for isdigit()[/QUOTE]isdigit() is already there, you just need to include it as shown in the diff.

@paulunderwood: Do you have some commands I can run? I'm trying to automate the build using a Docker container. So far, my Dockerfile is still quite modest:

[CODE]FROM gcc
RUN apt-get update && apt-get install -y libgmp-dev
COPY . /usr/src/openpfgw
WORKDIR /usr/src/openpfgw
RUN make
[/CODE]

paulunderwood 2017-02-08 22:09

I get libgmp.a from a recent installation of the latest GMP, and gwnum.a from compiling mprime -- so I guess you can wget it and make it to get GWNUM. HTH. :smile:

nordi 2017-02-08 22:36

I did get the libgmp.a, but cannot compile mprime:

[CODE]
/usr/src/mprime/linux64# make
[ ! -e ../security.h ] && touch ../security.h || true
[ ! -e ../security.c ] && touch ../security.c || true
[ ! -e ../secure5.c ] && touch ../secure5.c || true
gcc -I.. -I../gwnum -I/usr/local/include -DX86_64 -O2 -c prime.c
gcc -I.. -I../gwnum -I/usr/local/include -DX86_64 -O2 -c menu.c
make: *** No rule to make target 'factor64.o', needed by 'mprime'. Stop.
[/CODE]

paulunderwood 2017-02-08 22:42

Please try [c]make[/c] in the gwnum directory.

[code]/Downloads/p95/gwnum$ make -f make64
gcc -I.. -DX86_64 -O2 -c cpuid.c
gcc -I.. -DX86_64 -O2 -c gwnum.c
gcc -I.. -DX86_64 -O2 -c gwtables.c
gcc -I.. -DX86_64 -O2 -c gwthread.c
gcc -I.. -DX86_64 -O2 -c gwini.c
gcc -I.. -DX86_64 -O2 -c gwutil.c
g++ -I.. -I../qd -DX86_64 -O2 -c gwdbldbl.cpp
gcc -I.. -DX86_64 -O2 -c giants.c
gcc -I.. -DX86_64 -O2 -c ecmstag1.c
cp linux64/gwnum.a .
ar -rs gwnum.a cpuid.o gwnum.o gwtables.o gwthread.o gwini.o gwutil.o gwdbldbl.o giants.o ecmstag1.o
[/code]

nordi 2017-02-08 23:06

[QUOTE=paulunderwood;452563]Please try [c]make[/c] in the gwnum directory.[/QUOTE]
I also tried this, it also failed:

[CODE]/usr/src/mprime/gwnum# make -f make64
gcc -I.. -DX86_64 -O2 -c cpuid.c
gcc -I.. -DX86_64 -O2 -c gwnum.c
gcc -I.. -DX86_64 -O2 -c gwtables.c
gcc -I.. -DX86_64 -O2 -c gwthread.c
gcc -I.. -DX86_64 -O2 -c gwutil.c
g++ -I.. -I../qd -DX86_64 -O2 -c gwdbldbl.cpp
gcc -I.. -DX86_64 -O2 -c giants.c
gcc -I.. -DX86_64 -O2 -c ecmstag1.c
cp linux64/gwnum.a .
cp: cannot stat 'linux64/gwnum.a': No such file or directory
make64:19: recipe for target 'gwnum.a' failed
make: *** [gwnum.a] Error 1
[/CODE]Funny part is: There's not even a "linux64" directory in this directory. There's a linux64 directory at the top level of the project, but it also does not contain gwnum.a.

nordi 2017-02-08 23:13

Thanks for posting the example output! Turns out the repo on Github was missing a file. So far, it works now.

paulunderwood 2017-02-08 23:19

[QUOTE=nordi;452567]I also tried this, it also failed:

[CODE]/usr/src/mprime/gwnum# make -f make64
gcc -I.. -DX86_64 -O2 -c cpuid.c
gcc -I.. -DX86_64 -O2 -c gwnum.c
gcc -I.. -DX86_64 -O2 -c gwtables.c
gcc -I.. -DX86_64 -O2 -c gwthread.c
gcc -I.. -DX86_64 -O2 -c gwutil.c
g++ -I.. -I../qd -DX86_64 -O2 -c gwdbldbl.cpp
gcc -I.. -DX86_64 -O2 -c giants.c
gcc -I.. -DX86_64 -O2 -c ecmstag1.c
cp linux64/gwnum.a .
cp: cannot stat 'linux64/gwnum.a': No such file or directory
make64:19: recipe for target 'gwnum.a' failed
make: *** [gwnum.a] Error 1
[/CODE]Funny part is: There's not even a "linux64" directory in this directory. There's a linux64 directory at the top level of the project, but it also does not contain gwnum.a.[/QUOTE]

I just downloaded p95v2810.source.zip and [c]zipinfo[/c] gives

[CODE]
.....
drwx--- 6.3 fat 102 bx stor 16-Oct-30 14:42 gwnum/linux64/
-rw---- 6.3 fat 35851474 bx defN 16-Oct-30 10:33 gwnum/linux64/gwnum.a
-rw---- 6.3 fat 30960362 bx defN 16-Oct-30 10:32 gwnum/linux/gwnum.a
.....
[/CODE]

nordi 2017-02-08 23:55

@paulunderwood: I worked around the -lgcc_s problem you had. Remove the " -Wl,-no_pie" option from the makefile. It seems "ld" only has "-pie", not "-no_pie".

nordi 2017-02-09 00:59

I have a pretty well-automated version now: To build PFGW with Docker, get and extract the sources of PFGW. In the directory with the sources, create a Dockerfile with this content:
[CODE]FROM gcc
RUN apt-get update && apt-get install -y libgmp-dev wget unzip

RUN mkdir /usr/src/mprime && \
cd /usr/src/mprime && \
# openpfgw insists on version 28.6
wget -O mprime.zip --quiet https://www.mersenne.org/ftp_root/gimps/p95v286.source.zip && \
# unzip produces some warnings and therefore has exit code 1.
unzip mprime.zip ; \
rm mprime.zip
RUN cd /usr/src/mprime/gwnum && \
make -j -f make64

COPY . /usr/src/openpfgw
WORKDIR /usr/src/openpfgw
RUN cp /usr/lib/x86_64-linux-gnu/libgmp.a packages/gmp/64bit/
RUN cp /usr/src/mprime/gwnum/gwnum.a packages/gwnum/64bit/gwnum.a
RUN sed -i 's/-Wl,-no_pie//g' makefile

RUN make -j
[/CODE]Run "docker build -t pfgw ." (mind the dot at the end) to build a new image. Then "docker run -it pfgw" to start a container with that image. The executable is at /usr/src/openpfgw/pfgw64 and it actually works.

This assumes "#include <ctype.h>" fix to PFGW is already applied.

paulunderwood 2017-02-09 01:59

Thanks for the tips, nordi :tu:

Incidentally, I added [c]gwset_num_threads ( &gwdata, 4 );[/c] to every line with [c]gwinit2(&gwdata, sizeof(gwhandle), (char *) GWNUM_VERSION);[/c] and got a multi-threaded PFGW working at 300% on my Haswell 4770k. Good for speedy proofs, but not so good for overall number-crunching throughput :smile:

henryzz 2017-02-09 11:31

[QUOTE=paulunderwood;452579]Thanks for the tips, nordi :tu:

Incidentally, I added [c]gwset_num_threads ( &gwdata, 4 );[/c] to every line with [c]gwinit2(&gwdata, sizeof(gwhandle), (char *) GWNUM_VERSION);[/c] and got a multi-threaded PFGW working at 300% on my Haswell 4770k. Good for speedy proofs, but not so good for overall number-crunching throughput :smile:[/QUOTE]
It is more efficient the larger the FFT. There would be people using it if this feature was added to the main source. Please do rouge.
A larger fft can fit in the L3 cache with 1x4threads than 4x1threads

bur 2021-09-01 11:20

I tried compiling the sources from sourceforge, but get the following error:
[CODE]g++: error: packages/gwnum/64bit/gwnum.a: File or directory not found
make: *** [makefile:8: pfgw64] Error 1[/CODE]
The gwnum/64bit/ directory is indeed empty. Any idea how to get it to compile? Or is it unnceccessary anyway and I should just use the precompiled binaries? I have an i9-10900k.

rogue 2021-09-01 12:44

[QUOTE=bur;586976]I tried compiling the sources from sourceforge, but get the following error:
[CODE]g++: error: packages/gwnum/64bit/gwnum.a: File or directory not found
make: *** [makefile:8: pfgw64] Error 1[/CODE]
The gwnum/64bit/ directory is indeed empty. Any idea how to get it to compile? Or is it unnceccessary anyway and I should just use the precompiled binaries? I have an i9-10900k.[/QUOTE]

You have to d/l gwnum separately. This is part of Prime95, so d/l the Prime95 source. You will likely need to build the 64-bit Linux version of gwnum.a (if it isn't included) then copy to the directory specified above. There should be a linux makefile in the Prime95 sources.

Prime95 2021-09-01 17:13

[QUOTE=rogue;586983]You have to d/l gwnum separately. This is part of Prime95, so d/l the Prime95 source. You will likely need to build the 64-bit Linux version of gwnum.a (if it isn't included) then copy to the directory specified above. There should be a linux makefile in the Prime95 sources.[/QUOTE]

In the gwnum directory, type "make -f make64"

bur 2021-09-02 06:08

Ok, thanks, it worked.

Are there any flags I could use to optimize e.g. for avx2? The makefile didn't look like it, but I'm far from being an expert there.

rogue 2021-09-02 16:22

[QUOTE=bur;587042]Ok, thanks, it worked.

Are there any flags I could use to optimize e.g. for avx2? The makefile didn't look like it, but I'm far from being an expert there.[/QUOTE]

No. gwnum will choose the best FFT based upon the available hardware. If the best FFT uses AVX2, then it will use that FFT. If AVX2 is not available, then any FFT that requires AVX2 will not be used.

MJansen 2021-10-30 11:38

Hi I am looking for some help:

I have been using Ubuntu for a short while now and am not that advanced a user. I have been trying to get pfgw64 running and I succeeded in getting the source files from pfgw64 unzipped in the right place under Ubuntu, as well as GMP and Gwnum.

I found some links as to getting it running, but I guess the main find was an older website, since most of the suggested changes did not make a difference ([url]https://wiki.deac.wfu.edu/user/Software:OpenPFGW[/url]).

Running the make -f makefile command gives me the following error that I cannot fix myself:
[QUOTE]
~/Src/openpfgw$ make -f makefile
make -C pform/pflib
make[1]: Entering directory '/home/mydir/Src/openpfgw/pform/pflib'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/mydir/Src/openpfgw/pform/pflib'
make -C pform/pfmath
...
more of entering, nothing to be done, leaving messages
...
make[1]: Entering directory '/home/mydir/Src/openpfgw/pform/primesieve'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/mydir/Src/openpfgw/pform/primesieve'
g++ \
pform/pfgw/.libs/pfgw_main.a pform/pfio/.libs/pfio.a pform/pfoo/.libs/pfoo.a pform/pfgwlib/.libs/pfgwlib.a \
pform/pfmath/.libs/pfmath.a pform/pflib/.libs/pflib.a pform/primesieve/.libs/primesieve.a \
packages/gmp/64bit/libgmp.a packages/gwnum/64bit/gwnum.a -o pfgw64 -lpthread -lstdc++
/usr/bin/ld: pform/pfgw/.libs/pfgw_main.a(gw_prp.o): in function `bench_gwPRP(Integer*, unsigned int)':
gw_prp.cpp:(.text+0x471): undefined reference to `gwsquare2'
/usr/bin/ld: gw_prp.cpp:(.text+0x4b6): undefined reference to `gwsquare2_carefully'
/usr/bin/ld: pform/pfgw/.libs/pfgw_main.a(gw_prp.o): in function `prp_using_gwnum(Integer*, unsigned int, char const*, unsigned long*, int)':
gw_prp.cpp:(.text+0x702): undefined reference to `gwstartnextfft'
/usr/bin/ld: gw_prp.cpp:(.text+0x747): undefined reference to `gwsquare2'
/usr/bin/ld: gw_prp.cpp:(.text+0x909): undefined reference to `gwstartnextfft'
/usr/bin/ld: gw_prp.cpp:(.text+0x929): undefined reference to `gwsquare2_carefully'
/usr/bin/ld: pform/pfgw/.libs/pfgw_main.a(gf_factorize.o): in function `GFNDivisibilityTest(char const*, Integer*, char const*, Integer, int, int, int, int, Integer*)':
gf_factorize.cpp:(.text+0x139): undefined reference to `gwsquare2'
/usr/bin/ld: gf_factorize.cpp:(.text+0x214): undefined reference to `gwsquare2_carefully'
/usr/bin/ld: pform/pfio/.libs/pfio.a(pfscriptfns.o): in function `PFScriptFile::GWPowMod(Integer*, Integer, Integer, Integer, char*)':
pfscriptfns.cpp:(.text+0x464e): undefined reference to `gwmul'
/usr/bin/ld: pfscriptfns.cpp:(.text+0x46c7): undefined reference to `gwsquare2'
/usr/bin/ld: pfscriptfns.cpp:(.text+0x4726): undefined reference to `gwsquare2_carefully'
/usr/bin/ld: pfscriptfns.cpp:(.text+0x4748): undefined reference to `gwmul_carefully'
/usr/bin/ld: pform/pfoo/.libs/pfoo.a(algebra.o): in function `IntegerMultiplier::squaremulInteger(GWInteger&, int, int)':

... cut short
[/QUOTE]

So I guess the crux is in this: undefined reference to `gwsquare2', but where is this function and how do I get this function to be seen?

Kind regards
Michiel Jansen

paulunderwood 2021-10-30 12:56

i have just downloaded the latest OpenPFGW from sourceforge and it compiled for me straight way, although I did not put libgmp.a and gwnum.a in place so it did not complete.

I get:

[code]
g++: error: packages/gmp/64bit/libgmp.a: No such file or directory
g++: error: packages/gwnum/64bit/gwnum.a: No such file or directory
[/code]

which can be remedied by compiling gmp and mprime, and putting libgmp,a and gwnum.a in the right directories. :smile:

rogue 2021-10-30 12:56

[QUOTE=MJansen;592042]Hi I am looking for some help:

I have been using Ubuntu for a short while now and am not that advanced a user. I have been trying to get pfgw64 running and I succeeded in getting the source files from pfgw64 unzipped in the right place under Ubuntu, as well as GMP and Gwnum.

I found some links as to getting it running, but I guess the main find was an older website, since most of the suggested changes did not make a difference ([url]https://wiki.deac.wfu.edu/user/Software:OpenPFGW[/url]).

Running the make -f makefile command gives me the following error that I cannot fix myself:


So I guess the crux is in this: undefined reference to `gwsquare2', but where is this function and how do I get this function to be seen?

Kind regards
Michiel Jansen[/QUOTE]

It is likely not finding the library or the correct version of the gwnum library to link into the executable.

MJansen 2021-10-30 13:28

Quick reply. thanks for that!

[QUOTE=rogue;592045]It is likely not finding the library or the correct version of the gwnum library to link into the executable.[/QUOTE]

Reminds me: somewhere there was a hickup, i.e. makefile asked me for *.h files in the packages/gwnum/ directory instead of the packages/gwnum/64bit/ directory, I copied the *.h files from the sourceforge source there but not the created *.h files. I will look into that!

[Quote]i have just downloaded the latest OpenPFGW from sourceforge and it compiled for me straight way, although I did not put libgmp.a and gwnum.a in place so it did not complete.

I get:

Code:
g++: error: packages/gmp/64bit/libgmp.a: No such file or directory
g++: error: packages/gwnum/64bit/gwnum.a: No such file or directory
which can be remedied by compiling gmp and mprime, and putting libgmp,a and gwnum.a in the right directories. [/Quote]

If the previous mentioned try does not work, I will start anew clean from the source

(ps what file did you use, the snapshot from [url]https://sourceforge.net/p/openpfgw/code/HEAD/tree/[/url] or another file?)

Kind regards
Michiel

paulunderwood 2021-10-30 13:39

I used the latest snapshot.

Make sure you compile 64 bits for both gwnum and gmp.

MJansen 2021-10-30 14:02

[QUOTE=paulunderwood;592048]I used the latest snapshot.

Make sure you compile 64 bits for both gwnum and gmp.[/QUOTE]

I get the same error, but without the intermitting errors after a clean re install. I thought that it could be caused by two files that were placed in the packages/gmp/64bit directory that I had not copied the last time (have been struggling with GMP for a while, before I finally succeeded in compiling it under Ubuntu and it is working fine now for some time in 64 bits).

So my guess is, the problem should be in the gwnum.a file. I copied the source file from here:
[url]http://www.mersenne.org/ftp_root/gimps/p95v303b6.source.zip[/url]

Or should I use the ready made Linux version:
[url]http://www.mersenne.org/ftp_root/gimps/p95v303b6.linux64.tar.gz[/url]

Kind regards
Michiel

MJansen 2021-10-30 14:17

[QUOTE=MJansen;592050]I get the same error, but without the intermitting errors after a clean re install. I thought that it could be caused by two files that were placed in the packages/gmp/64bit directory that I had not copied the last time (have been struggling with GMP for a while, before I finally succeeded in compiling it under Ubuntu and it is working fine now for some time in 64 bits).

So my guess is, the problem should be in the gwnum.a file. I copied the source file from here:
[url]http://www.mersenne.org/ftp_root/gimps/p95v303b6.source.zip[/url]

Or should I use the ready made Linux version:
[url]http://www.mersenne.org/ftp_root/gimps/p95v303b6.linux64.tar.gz[/url]

Kind regards
Michiel[/QUOTE]

To answer my own question: use the source ;-)
It ran through the make process without hick-ups this time round!

Note: use the source linked here, not the P95v306b4.source.zip I used earlier ...

Thanks for the help!
Kind regards
Michiel

paulunderwood 2021-10-30 14:21

To get gwnum.a I compiled with [C]make -f make64[/C] in Prime95's gwnum directory from your download link. Then I copied with [C]cp gwnum.a ~/Downloads/openpfgw-code-r669/packages/gwnum/64bit[/C].

For gmp I just did [C]cp /usr/lib/x86_64-linux-gnu/libgmp.a ~/Downloads/openpfgw-code-r669/packages/gmp/64bit[/C].

pfgw64 compiled fully.

MJansen 2021-10-30 14:25

[QUOTE=paulunderwood;592053]To get gwnum.a I compiled with [C]make -f make64[/C] in Prime95's gwnum directory from your download link. Then I copied with [C]cp gwnum.a ~/Downloads/openpfgw-code-r669/packages/gwnum/64bit[/C].

For gmp I just did [C]cp /usr/lib/x86_64-linux-gnu/libgmp.a ~/Downloads/openpfgw-code-r669/packages/gmp/64bit[/C].

pfgw64 compiled fully.[/QUOTE]

Crosspost ;-) found it just before you posted, thanx again!
Kund regards
Michiel

paulunderwood 2021-10-30 14:26

[QUOTE=MJansen;592054]Crosspost ;-) found it just before you posted, thanx again!
Kund regards
Michiel[/QUOTE]

You're welcome! :tu:

MJansen 2021-10-30 14:40

And I just found my first PRP under Ubuntu using pfgw64, so it works fine!

For the other starters out there that wonder how to run pfgw64 under Ubuntu:
Make a .txt file that has some prime candidates, say:
10000455041*3607#/210-2
10000455041*3607#/210-47298

and save it in the same directory as pfgw64 under the name test3607.txt

In the terminal type
./pfgw64 test3607.txt [enter]

And a PRP-3 is found ;-)


All times are UTC. The time now is 23:07.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.