Building 14.1/autoconf on Solaris 11/SPARC

olegkirillov · 2017-06-10, 12:26

Hi, guys.
I have an Oracle T4-4 system and I want to put it on a CPU/memory stress-test with a some more benefit than raising the environment temperature a few Celcius higher.

But I tried to build several Mlucas sources with no success. The basic README provides no instructions to build on a SPARC at all, so I had to figure out that I should not use threads and optimizations at all just to compile at least some modules. But not all.

Finally I reverted to the latest version (http://www.mersenneforum.org/mayer/s...as-14.1.tar.gz) and tried to get the most from GCC (I have no Sun's CC), but build end up with an output attached. See no direction to progress, need some clues.

olegkirillov · 2017-06-10, 16:35

Update:
managed to attempt building with SUN Pro cc, yet have no result.

ewmayer · 2017-06-10, 22:08

Hi, Oleg:

The automake stuff has never been tested on your kind of platform - congrats on being the first. :)

I would suggest eschewing automake for now, and just trying a basic command-line compile. Mlucas has no Sparc SIMD support, so we're stuck with the basic scalar-double C code build. Parallel builds using pthreads should work on your platform, but for starters let's stick to single-threaded. cd to the src-dir inside your unzipped tarball, 'mkdir obj' within src, then do this:

cd obj
gcc -c -O3 ../*.c >& build.log
grep -i error build.log
[Assuming above grep comes p empty] gcc -o Mlucas *.o -lm -lrt

If that works, try a simple single FFT length set of self-tests:

./Mlucas -fftlen 1152 -iters 100

Keep an eye out for errors during that set of tests - post them here if you encounter any - and when done post the resulting mlucas.cfg file contents.

If that all works, feel free to proceed to a || build, preferably in a separate obj_pthr dir, again immediately beneath src;

cd obj_pthr
gcc -c -O3 -DUSE_THREADS ../*.c >& build.log
grep -i error build.log
[Assuming above grep comes p empty] gcc -o Mlucas *.o -lm -lrt -lpthread

olegkirillov · 2017-06-10, 22:50

Code:

"../twopmodq80.c", line 2957: warning: shift count negative or too big: >> -50
../twopmodq96.c:
../types.c:
../util.c:
"../util.c", line 1071: undefined symbol: OS_BITS
cc: acomp failed for ../util.c

root@solt44:~/dev/mlucas-14.1/src/obj# uname -a
SunOS solt44.local 5.11 11.3 sun4v sparc sun4v

Alas!

ewmayer · 2017-06-10, 23:23

Quote:

Originally Posted by olegkirillov

Code:

"../twopmodq80.c", line 2957: warning: shift count negative or too big: >> -50
../twopmodq96.c:
../types.c:
../util.c:
"../util.c", line 1071: undefined symbol: OS_BITS
cc: acomp failed for ../util.c

root@solt44:~/dev/mlucas-14.1/src/obj# uname -a
SunOS solt44.local 5.11 11.3 sun4v sparc sun4v

Alas!

Don't despair, you're actually quite close. The twomodq80 warning is due to compiler optimizing the various clauses of an if/then/else in a macro in an attempt to permit speculative "take both branches but then discard the one whose arg value-clause is actually false based on the runtime value of the datum in question", i.e. is ignorable.

The util.c error means the platform.h file has no preprocessor clause as yet which correctly identifies the OS-bitness on your platform.

And looking at the v14.1 platform.h file, I see the problem - find the __sparc clause below in your copy of the file, and then insert the bolded text as shown:

Code:

#elif(defined(__sparc))

	#define CPU_TYPE
	#define CPU_IS_SPARC

	#if __SIZEOF_POINTER__ == 4
		#define OS_BITS 32
	#elif __SIZEOF_POINTER__ == 8
		#define OS_BITS 64
	#else
		#error __SIZEOF_POINTER__ defined but returns unrecognized value! Use gcc -dM -E < /dev/null | grep __SIZEOF_POINTER__ to examine.
	#endif

	#define MUL64_SUCKS

I will add a similar snip wrapped inside an #ifndef OS_BITS at bottom of the platform.h file, to catch any other such cases where more targeted OS_BITS setting is not done for the platform in question.

And, he, he, don't take the name of the bottommost #def in the above snip personally - it was true in all the early generations of Sparc, probably no longer is, but only matters for trial-factoring, which is not supported in production builds of Mlucas anyway.

olegkirillov · 2017-06-11, 06:43

It returned that __SIZEOF_POINTER__ = 4 and caught the error you defined. Quite strange.

Therefore I just left #define OS_BITS 64 as I know it and compiled the code. But linking fails:

Code:

root@solt44:~/dev/mlucas-14.1/src/obj# cc -o Mlucas *.o -lm -lrt
Undefined                       first referenced
 symbol                             in file
given_N_get_maxP                    gcd_lehmer.o
get_fft_radices                     gcd_lehmer.o
get_default_fft_length              Mlucas.o
test_fft_radixtables                gcd_lehmer.o
get_nextlarger_fft_length           Mlucas.o
ld: fatal: symbol referencing errors
root@solt44:~/dev/mlucas-14.1/src/obj#

ewmayer · 2017-06-11, 20:34

Quote:

Originally Posted by olegkirillov

It returned that __SIZEOF_POINTER__ = 4 and caught the error you defined. Quite strange.

Therefore I just left #define OS_BITS 64 as I know it and compiled the code. But linking fails:

Code:

root@solt44:~/dev/mlucas-14.1/src/obj# cc -o Mlucas *.o -lm -lrt
Undefined                       first referenced
 symbol                             in file
given_N_get_maxP                    gcd_lehmer.o
get_fft_radices                     gcd_lehmer.o
get_default_fft_length              Mlucas.o
test_fft_radixtables                gcd_lehmer.o
get_nextlarger_fft_length           Mlucas.o
ld: fatal: symbol referencing errors
root@solt44:~/dev/mlucas-14.1/src/obj#

That is weird with the sizeof returning 32-bit - what does __SIZEOF_LONG_LONG__ return on your system?

Re. the link errors, that looks like get_fft_radices.o is missing. Do you see that object file in your obj-dir? If not, what happens when you try to compile just that .c file?

Edit: Allow me to anticipate the source of the above missing-obj-file issue - you need to download the patched version of said file (which had a duplicate-case-value bug for non-SIMD builds) under the 07 Feb 2016 date on the README page. I should've rolled the patch in with the main tarball, but failed to do so. Little point now, with v17 imminent.

olegkirillov · 2017-06-12, 10:13

__SIZEOF_LONG_LONG__ = 8

Got it compiled and linked using the patch, you were right.

But building with threads fails:

Code:

root@solt44:~/dev/mlucas-14.1/src/obj# gcc -c -O3 -DUSE_THREADS ../*.c

...

In file included from ../types.h:30:0,
                 from ../util.h:30,
                 from ../util.c:23:
../platform.h:1043:26: fatal error: sys/sysctl.h: No such file or directory
   #include <sys/sysctl.h>
                          ^
compilation terminated.
root@solt44:~/dev/mlucas-14.1/src/obj-pthr#

Code:

root@solt44:~/dev/mlucas-14.1/src/obj# cc -c -O3 -DUSE_THREADS ../*.c

...

../util.c:
"../platform.h", line 1074: #error: Multithreading currently only supported for Linux/GCC builds!
cc: acomp failed for ../util.c
root@solt44:~/dev/mlucas-14.1/src/obj-pthr#

The built binary creates only one thread taking 0.7% of CPU load. The system is capable of servicing up to 256 threads.

Code:

root@solt44:~/dev/mlucas-14.1/src/obj# ./Mlucas -s m -iters 1000 -nthread 128

    Mlucas 14.1

    http://hogranch.com/mayer/README.html

INFO: testing qfloat routines...
CPU Family = Sparc, OS = SunOS / Solaris, 64-bit Version, compiled with SunPro C, Version [Unknown].
INFO: Using prefetch.
INFO: Using inline-macro form of MUL_LOHI64.
INFO: MLUCAS_PATH is set to ""
INFO: using 53-bit-significand form of floating-double rounding constant for scalar-mode DNINT emulation.
INFO: testing IMUL routines...
INFO: testing FFT radix tables...
Multithreading not enabled; ignoring -nthread argument.
           Mlucas selftest running.....

/****************************************************************************/

M20000047: using FFT length 1024K = 1048576 8-byte floats.
 this gives an average   19.073531150817871 bits per digit
Using complex FFT radices      1024        16        32

ewmayer · 2017-06-12, 20:58

Hi, Oleg:

The thread-related headers other than pthread.h are needed to support 2 things: [1] thread affinity setting (cf. threadpool.c for the mechanics of this for various posix platforms I currently run code on), and [2] the Posix nanosleep() function. Those are surely supported by Sparc Linux (or whichever Posix-compliant OS you are using), but you likely need to customize the includes to grab a different set of header files appropriate for your platform. Perhaps consulting whatever pthread-related documentation there is for your platform and/or asking a colleague who has dealt with multithreaded coding for it should be the next step?

In the meantime, would you be kind enough to do the standard self-tests - I suggest using './Mlucas -s m >& selftest.log' so we can check the resulting screen outputs for possible errors (e.g. at ertain FFT lengths or for certain FFT radices, often related to overaggressive compiler optimizations) using your single-threaded binary and post the resulting build.log and mlucas.cfg file, along with a summary of the basic processor info? (I use cat /proc/cpuinfo under Linux, and only need the data for proc 0, not all the other identical cores.)

chris2be8 · 2017-06-13, 16:04

Quote:

Originally Posted by olegkirillov

The built binary creates only one thread taking 0.7% of CPU load. The system is capable of servicing up to 256 threads.

I used to support Sun servers (among other things) before I retired. One type took hyperthreading to it's logical conclusion by having 2 real cores, but supporting 128 threads per core. If you have a system like that you will find that using more than 2 threads won't increase total throughput much. Try running 2 or more copies at once and see how throughput behaves before spending a lot of time getting pthreads working.

I can't remember exact details and don't have access to them any more to check.

Chris

ET_ · 2017-06-14, 13:41

Quote:

Originally Posted by ewmayer

That is weird with the sizeof returning 32-bit - what does __SIZEOF_LONG_LONG__ return on your system?

Re. the link errors, that looks like get_fft_radices.o is missing. Do you see that object file in your obj-dir? If not, what happens when you try to compile just that .c file?

Edit: Allow me to anticipate the source of the above missing-obj-file issue - you need to download the patched version of said file (which had a duplicate-case-value bug for non-SIMD builds) under the 07 Feb 2016 date on the README page. I should've rolled the patch in with the main tarball, but failed to do so. Little point now, with v17 imminent.

A jump of 3 versions?!?

Nevermind, I found the answer...

Code:

First, a note on version numbering: After many years frozen at 3.0x (since my
x86 code was until recently wildly uncompetitive with Prime95), now that I'm
< 2x slower (on Haswell+), resumed version numbering according to the scheme:

Major index = year - 2000
Minor index = release # of that year, zero-indexed.

As before, a patch suffix of x, y, or z following the numeric index indicates
an [alpha,beta,gamma] (experimental,unstable) code. Since I consider this
release stable and it's the 2nd release of the year
, thus 14.1 = [20]14.[--2][no xyz suffix].

2017-06-10, 22:50	#4
olegkirillov Jun 2017 5 Posts	Code: "../twopmodq80.c", line 2957: warning: shift count negative or too big: >> -50 ../twopmodq96.c: ../types.c: ../util.c: "../util.c", line 1071: undefined symbol: OS_BITS cc: acomp failed for ../util.c root@solt44:~/dev/mlucas-14.1/src/obj# uname -a SunOS solt44.local 5.11 11.3 sun4v sparc sun4v Alas!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
AMD64 on Solaris	Kyle	Software	9	2012-11-26 13:27
autoreconf (from autoconf 2.68) doesn't work...	Ralf Recker	GMP-ECM	1	2011-07-25 14:14
Mlucas on Sparc -	Unregistered	Mlucas	0	2009-10-27 20:35
mprime on SunOS/sparc?	rudi_m	Software	17	2005-12-26 23:35
Solaris 10	moo	Software	0	2004-12-01 01:56

2017-06-10, 22:08	#3
ewmayer ∂²ω=0 Sep 2002 República de California 5·2,351 Posts	Hi, Oleg: The automake stuff has never been tested on your kind of platform - congrats on being the first. :) I would suggest eschewing automake for now, and just trying a basic command-line compile. Mlucas has no Sparc SIMD support, so we're stuck with the basic scalar-double C code build. Parallel builds using pthreads should work on your platform, but for starters let's stick to single-threaded. cd to the src-dir inside your unzipped tarball, 'mkdir obj' within src, then do this: cd obj gcc -c -O3 ../.c >& build.log grep -i error build.log* [Assuming above grep comes p empty] gcc -o Mlucas .o -lm -lrt If that works, try a simple single FFT length set of self-tests: ./Mlucas -fftlen 1152 -iters 100 Keep an eye out for errors during that set of tests - post them here if you encounter any - and when done post the resulting mlucas.cfg file contents. If that all works, feel free to proceed to a \|\| build, preferably in a separate obj_pthr dir, again immediately beneath src; cd obj_pthr gcc -c -O3 -DUSE_THREADS ../.c >& build.log grep -i error build.log [Assuming above grep comes p empty] gcc -o Mlucas *.o -lm -lrt -lpthread

2017-06-12, 20:58	#9
ewmayer ∂²ω=0 Sep 2002 República de California 26753₈ Posts	Hi, Oleg: The thread-related headers other than pthread.h are needed to support 2 things: [1] thread affinity setting (cf. threadpool.c for the mechanics of this for various posix platforms I currently run code on), and [2] the Posix nanosleep() function. Those are surely supported by Sparc Linux (or whichever Posix-compliant OS you are using), but you likely need to customize the includes to grab a different set of header files appropriate for your platform. Perhaps consulting whatever pthread-related documentation there is for your platform and/or asking a colleague who has dealt with multithreaded coding for it should be the next step? In the meantime, would you be kind enough to do the standard self-tests - I suggest using './Mlucas -s m >& selftest.log' so we can check the resulting screen outputs for possible errors (e.g. at ertain FFT lengths or for certain FFT radices, often related to overaggressive compiler optimizations) using your single-threaded binary and post the resulting build.log and mlucas.cfg file, along with a summary of the basic processor info? (I use cat /proc/cpuinfo under Linux, and only need the data for proc 0, not all the other identical cores.)