ggnfs improvements

[wombatman]

Originally Posted by wreck

When I try your gnfs-lasieve4I16e on number R1340L (an degree 8 snfs) near q=316M, the speed
is 0.73 sec/rel, while the original binary is 0.96 sec/rel.

So I try to compile a Linux or a Windows binary myself but failed.

using your method, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
icc -O3 -march=native -mtune=native -funroll-loops -static-intel -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
make：icc：命令未找到
make: *** [Makefile:82：asm/lasched11.o] 错误 127

using cc to compile, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
cc -O3 -march=native -mtune=native -funroll-loops -static-intel -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
cc: error: unrecognized command line option ‘-static-intel’; did you mean ‘-static-pie’?
make: *** [Makefile:82：asm/lasched11.o] 错误 1

change -static-intel to -static-pie, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
cc -O3 -march=native -mtune=native -funroll-loops -static-pie -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
lasched.w: In function ‘lasched’:
lasched.w:92:19: warning: implicit declaration of function ‘_mm512_loadu_epi32’; did you mean ‘_mm512_load_epi32’? [-Wimplicit-function-declaration]
lasched.w:92:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:93:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:193:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:194:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
make: *** [Makefile:81：asm/lasched11.o] 错误 1

Any suggestion? and if possible , could you compile a Windows binary of gnfs-lasieve4I16e.exe?

What CPU do you have, and is it AVX512 compatible?

[bsquared]

Originally Posted by wreck

When I try your gnfs-lasieve4I16e on number R1340L (an degree 8 snfs) near q=316M, the speed
is 0.73 sec/rel, while the original binary is 0.96 sec/rel.

So I try to compile a Linux or a Windows binary myself but failed.

using your method, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
icc -O3 -march=native -mtune=native -funroll-loops -static-intel -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
make：icc：命令未找到
make: *** [Makefile:82：asm/lasched11.o] 错误 127

using cc to compile, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
cc -O3 -march=native -mtune=native -funroll-loops -static-intel -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
cc: error: unrecognized command line option ‘-static-intel’; did you mean ‘-static-pie’?
make: *** [Makefile:82：asm/lasched11.o] 错误 1

change -static-intel to -static-pie, I met

chenbo@chenbo:~/D/chenbo/my/math/cpp/yafu/yafu533/yafu-master/factor/lasieve4_64$ make all AVX512_LASCHED=1 AVX512_LASETUP=1 AVX512_TD=1 LASTATS=1
cc -O3 -march=native -mtune=native -funroll-loops -static-pie -DAVX512_LASIEVE_SETUP -DAVX512_LASCHED -DAVX512_TD -DGATHER_STAT -I. -I./asm  -I/usr/local/include -I../../include -c -DI_bits=11 -o asm/lasched11.o asm/lasched.c
lasched.w: In function ‘lasched’:
lasched.w:92:19: warning: implicit declaration of function ‘_mm512_loadu_epi32’; did you mean ‘_mm512_load_epi32’? [-Wimplicit-function-declaration]
lasched.w:92:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:93:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:193:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
lasched.w:194:19: error: incompatible types when initializing type ‘__m512i’ {aka ‘__vector(8) long long int’} using type ‘int’
make: *** [Makefile:81：asm/lasched11.o] 错误 1

Any suggestion? and if possible , could you compile a Windows binary of gnfs-lasieve4I16e.exe?

As of right now I can only guarantee it will build with Intel's compiler but I will try to fix that later after all the optimization is done.

Windows build also is next on the list.

If you can't build it then I'm confused how you got the improved timing that you posted. Did the Linux binary in the git repository work?

[wreck]

My computer's CPU is Intel Core i5-1135G7, the /proc/cpuinfo says it support avx512.

After I install icc , I could compile a gnfs-lasieve4I16e binary successfully,
but a little slower (0.75062 sec/rel).

The previous binary I run is downloaded from github
https://github.com/bbuhrow/yafu
the 533 times commit also the latest one, under the folder
yafu-master/factor/lasieve4_64
there are 6 binary files named gnfs-lasieve4I11e, ..., gnfs-lasieve4I16e
the last one is I test, and its speed is 0.74686 sec/rel.
I could run them under my Ubuntu 20.04 system (after adding 777 permission).

I test again under windows 10 system using the same command,
this time the windows binary's speed is 0.87038 sec/rel.
The Linux avx512 version is still 15% faster.

I'll try later to compile an icc compiler under my windows 10 system
using cygwin64.

I ready to use the binary you provide under Linux system to sieve.

Good job!

[bsquared]

Originally Posted by wreck

My computer's CPU is Intel Core i5-1135G7, the /proc/cpuinfo says it support avx512.

After I install icc , I could compile a gnfs-lasieve4I16e binary successfully,
but a little slower (0.75062 sec/rel).

The previous binary I run is downloaded from github
https://github.com/bbuhrow/yafu
the 533 times commit also the latest one, under the folder
yafu-master/factor/lasieve4_64
there are 6 binary files named gnfs-lasieve4I11e, ..., gnfs-lasieve4I16e
the last one is I test, and its speed is 0.74686 sec/rel.
I could run them under my Ubuntu 20.04 system (after adding 777 permission).

I test again under windows 10 system using the same command,
this time the windows binary's speed is 0.87038 sec/rel.
The Linux avx512 version is still 15% faster.

I'll try later to compile an icc compiler under my windows 10 system
using cygwin64.

I ready to use the binary you provide under Linux system to sieve.

Good job!

I'm glad it is working for you, and I'm glad that the binaries in the repository are working! I complied them statically, but got a warning that -cilkrts was not available statically and was linked dynamically. But maybe the library is unused or maybe you already have it in your library search path.

Has anyone else tried the pre-compiled linux binaries and had them work?

I admit I'm surprised there aren't more people here with AVX512 capable computers that are trying this.

[pinhodecarlos]

Thank you Ben for this. I'm pending Greg's position for NFS@Home to have a friend trial it.

[kruoli]

The precompiled 15e siever is now runnig for over two hours on Debian 11.5...
./gnfs-lasieve4I15e -v -f 100000000 -c 1000 -r x.job
This should not take that long, right? It is a job file from NFS@home.

[bsquared]

Originally Posted by kruoli

The precompiled 15e siever is now runnig for over two hours on Debian 11.5...
./gnfs-lasieve4I15e -v -f 100000000 -c 1000 -r x.job
This should not take that long, right? It is a job file from NFS@home.

With -c 1000, probably not. If you want to send the job file I can test here. (IM, post, whatever you prefer)

[kruoli]

Thank you!

$ cat x.job
n: 2097026643802877507269943929206861145366651524876715588205870797295371435262177200032311581577065062943201951040730730939818168279185975155039206178294169390003508531914739075399798610623509784910133338777
skew: 0.26122
type: snfs
size: 262
c6: 14661
c0: -1
Y0: -8727963568087712425891397479476727340041449
Y1: 1
rlim: 134000000
alim: 266000000
lpbr: 32
lpba: 32
mfbr: 94
mfba: 64
rlambda: 3.9
alambda: 2.7

[bsquared]

Seems to be working for me, so the precompiled binary probably needs the dynamic library -cilkrts. But I'm surprised it doesn't just error out for you. You could try the opencilk git repository.

Here's what I get:

time ./gnfs-lasieve4I15e -v -f 100000000 -c 1000 -r snfs_100000k_I15.txt
gnfs-lasieve4I15e (with asm64,avx-512 mmx-td,avx-512 lasetup,avx-512 lasched,avx-512 sieve1): L1_BITS=15
Warning:  lowering FB_bound to 99999999.
FBsize 14496980+0 (deg 6), 5761454+0 (deg 1)
total yield: 3248, q=100001029 (0.11645 sec/rel) ETA 0h00m)
54 Special q, 422 reduction iterations
reports: 132797745->29057200->27213568->21932573->8129900->4480475
Number of relations with k rational and l algebraic primes for (k,l)=:

Total yield: 3248
0/0 mpqs failures, 44983/2850 vain mpqs
milliseconds total: Sieve 66710 Sched 110120 medsched 4950
TD 117090 (Init 3730, MPQS 62140) Sieve-Change 40, lasieve_setup 79330
TD side 0: init/small/medium/large/search: 1540 10910 1100 8060 10180
sieve: init/small/medium/large/search: 1220 16450 1090 11410 4920
TD side 1: init/small/medium/large/search: 820 7180 890 5550 4550
sieve: init/small/medium/large/search: 1210 19470 1050 8430 1460
421.525u 4.794s 7:12.06 98.6%   0+0k 0+736io 0pf+0w

Here's the original for comparison:

time ./gnfs-lasieve4I15e-mpqs -v -f 100000000 -c 1000 -r snfs_100000k_I15.txt
gnfs-lasieve4I15e (with asm64): L1_BITS=15, SVN $Revision: 430 $
Warning:  lowering FB_bound to 99999999.
FBsize 14496980+0 (deg 6), 5761454+0 (deg 1)
total yield: 3357, q=100001029 (0.17463 sec/rel) ETA 0h00m)
54 Special q, 422 reduction iterations
reports: 132797299->29063205->27210474->21929840->8129762->4480483
Number of relations with k rational and l algebraic primes for (k,l)=:

Total yield: 3357
0/0 mpqs failures, 44987/87347 vain mpqs
milliseconds total: Sieve 66740 Sched 132560 medsched 12840
TD 147830 (Init 3470, MPQS 93390) Sieve-Change 30, lasieve_setup 226240
TD side 0: init/small/medium/large/search: 2170 10000 1090 7830 9740
sieve: init/small/medium/large/search: 1260 17260 1110 11360 4220
TD side 1: init/small/medium/large/search: 1560 7200 1050 5340 4460
sieve: init/small/medium/large/search: 1060 19700 840 8760 1170
630.200u 4.517s 10:35.57 99.8%  0+0k 784+760io 1pf+0w

Tinyecm is missing some of the 3LP's. I should look into putting mpqs back in there as a fallback if tinyecm fails to factor.

[frmky]

Originally Posted by pinhodecarlos

Thank you Ben for this. I'm pending Greg's position for NFS@Home to have a friend trial it.

That might take a while. As far as I know, the BOINC scheduler doesn't yet support AVX-512. There's an open issue on it from three years ago, https://github.com/BOINC/boinc/issues/3180
And I think only Primegrid is using it currently, and they use a client that detects it at runtime rather than using the BOINC scheduler to send a different binary.

[pinhodecarlos]

Speak with David. There's a way to overwrite that. We used to use what the boinc world calls "optimised" clients (Enigma project, etc). You just need to set up your side to not detect client version.