mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Mlucas (https://www.mersenneforum.org/forumdisplay.php?f=118)
-   -   Mlucas v19.1 available (https://www.mersenneforum.org/showthread.php?t=26483)

ewmayer 2021-02-12 00:00

Mlucas v19.1 available
 
[url=http://www.mersenneforum.org/mayer/README.html]Mlucas v19.1 has gone live[/url]. Use this thread to report bugs, build issues, and for any other related discussion.

The only "must have" reason for folks already using v19 is in case you want to also play with builds using the Clang/LLVM compiler on Armv8-SIMD - supporting CPUs, including the new Apple Silicon ones, for which Clang is the native compiler.

Special thanks to forumites ldesnogues for reporting the "v19 won't build using Clang on Armv8" issue and helping to diagnose its root cause. There is also a nifty auto-install-and-tune-for-your-hardware script which author tdulcet has been tweaking based on my feedback and lots of back-and-forth; thanks also to tdulcet and dan2 for the bigly-enhanced version of the primenet.py work-management script which builds on Loïc Le Loarer's Primenet-API-interfacing work of last year. Details, as always at the above README page.

I have tried to make the README, which is quite sprawling due to the need to support the do-it-yourselfers who are legion in Linuxworld, more easily navigable - I hope the added "jump to"-arrowed in-page-links make it somewhat easier to navigate between e.g. the release description and how-to-build sections.

I expect to be spending the coming week or so uploading patches - hopefully more to the README than the v19.1 sources - and doing a big honking file merge of the v19.1 code changes upward into the v20 development branch, which has lain dormant for the past 2 months. All the v19.1 effort would have been needed in v20 anyway, so the time has not been wasted, I'm simply looking forward to getting back to feature-add work in form of p-1 factoring support.

Please subscribe to this thread if you want to be notified of patch uploads.

ewmayer 2021-02-12 20:29

README updated with corrected wget for tdulcet's mlucas.sh install script - original version would download and run it immediately, now users can first parse it, comment out the primenet.py - invocation block if they want to "try before you buy", etc.

I'm also looking for an owner of a hybrid BIG/little-CPU system like Odroid N2 to test tdulcet's Mlucas v19.1 install/autotune script on such. With my guidance he's made several changes in an effort to support such systems, which typically need separate run directories for each CPU, each with an mlucas.cfg file containing FFT params properly tuned for the CPU in question.

Dylan14 2021-02-15 20:35

I've updated the PKGBUILD for Arch Linux to v19.1, which follows the procedure as described in the readme document.

The fp-link patch is no longer needed, however, the sysctl-missing patch is still needed.

ewmayer 2021-02-15 21:58

[QUOTE=Dylan14;571681]I've updated the PKGBUILD for Arch Linux to v19.1, which follows the procedure as described in the readme document.

The fp-link patch is no longer needed, however, the sysctl-missing patch is still needed.[/QUOTE]

Thanks - the latter is the sysctl-deprecated warnings? Handling those is on my v20 to-do list - it's not as simple as blanket-removing the includes from platform.h, because I always try to support older platforms within reason, so that needs proper preprocessor #ifdef wrapping to retain the include on older distros of Linux and MacOS where that header is needed.

You're getting warnings, or your version of GCC is treating those "deprecated"s as errors?

Dylan14 2021-02-15 23:02

When I try to build on Arch Linux (which is presently on kernel version 5.10.16) the error I would get if I kept the include <sys/sysctl.h> is:

[code]platform.h:1307:12: fatal error: sys/sysctl.h: No such file or directory
compilation terminated.
[/code]This is using gcc 10.2.0.
This would not be needed, if I was using the linux-lts kernel which is on version 5.4 and has the sysctl.h file - so doing my blanket patch is a bit risky - I should only run the patch if the kernel version is at least 5.5.

ewmayer 2021-02-15 23:40

[QUOTE=Dylan14;571694]When I try to build on Arch Linux (which is presently on kernel version 5.10.16) the error I would get if I kept the include <sys/sysctl.h> is:

[code]platform.h:1307:12: fatal error: sys/sysctl.h: No such file or directory
compilation terminated.
[/code]This is using gcc 10.2.0.
This would not be needed, if I was using the linux-lts kernel which is on version 5.4 and has the sysctl.h file - so doing my blanket patch is a bit risky - I should only run the patch if the kernel version is at least 5.5.[/QUOTE]

What is needed is some way of conditionally including the file only on OS/kernel combinations which support it. I dumped all the compiler predefines for one of my Ubuntu v19 systems, 'uname -a ' indicates it's kernel 5.3:
[i]
Linux ewmayer-NUC8i3CYS 5.3.0-59-generic #53-Ubuntu SMP Wed Jun 3 15:52:15 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
gcc version 9.2.1 20191008 (Ubuntu 9.2.1-9ubuntu2)
[/i]
...but I don't see any specific-Linux-version info in the GCC predefines.

Do me a favor - for your Arch Linux distro, cd to mlucas_v19.1/src and run the following command there:
[i]
gcc -dM -E align.h < /dev/null > predefs.txt
[/i]
(The align.h header is just so both the system and Mlucas predefs get dumped). Attach the resulting predefs.txt file to a post.

Dylan14 2021-02-15 23:50

1 Attachment(s)
[QUOTE=ewmayer;571696]
Do me a favor - for your Arch Linux distro, cd to mlucas_v19.1/src and run the following command there:
[i]
gcc -dM -E align.h < /dev/null > predefs.txt
[/i]
(The align.h header is just so both the system and Mlucas predefs get dumped). Attach the resulting predefs.txt file to a post.[/QUOTE]

See attached file.

Dylan14 2021-02-16 00:24

Okay, I have figured out a way to determine when to patch within the PKGBUILD, as seen here in the prepare function:

[CODE]prepare() {
cd "${srcdir}"/"${pkgname}"_v"${pkgver}"
#Only patch if the kernel version is at least 5.5.0
kermajver=`uname -r | cut -d. -f1`
kerminver=`uname -r | cut -d. -f2`
if [ $kermajver -gt 5 ]; then
patch -p1 < "../../sysctl-missing.patch"
elif [ $kermajver -eq 5 ] && [ $kerminver -ge 5 ]; then
patch -p1 < "../../sysctl-missing.patch"
fi
}[/CODE]Basically, if the kernel major version is greater then 5 then run the patch, or if the kernel major version is 5 and the minor version is at least 5 then also run the patch. Otherwise do nothing.

ewmayer 2021-02-16 20:02

Thanks for the predefs - looking through those triggered a recollection, that I'd previously made a note-to-self for the v20 release re. this, namely that the sysctl.h deprecation was tied not to a specific Linux kernel version, but rather to the GLIBC version. That's useful because GCC predefines don't have info re. the former, but do have the GLIBC version info.

Per this [url=https://github.com/open5gs/open5gs/issues/600]Github discussion[/url], "sysctl() is deprecated and may break build with glibc >= 2.30", so we wrap the #include like so:
[code]#if (__GLIBC__ < 2) || (__GLIBC_MINOR__ < 30)
#warning GLIBC either not defined or version < 2.30 ... including <sys/sysctl.h> header.
#include <sys/sysctl.h>
#endif[/code]
If you make that mod in your platform.h file, does that fix the problem without having to apply your patch?

Dylan14 2021-02-16 20:13

[QUOTE=ewmayer;571770]...
If you make that mod in your platform.h file, does that fix the problem without having to apply your patch?[/QUOTE]

That fixed the issue.

Lorenzo 2021-02-17 20:31

Hello!) I just want to share my experience with Apple M1 CPU.

Compiled smoothly without issues (i have included -DUSE_ARM_V8_SIMD flag according with the README page).

[CODE]CPU Family = ARM Embedded ABI, OS = OS X, 64-bit Version, compiled with Gnu-C-compatible [llvm/clang], Version 12.0.0 (clang-1200.0.32.29).
INFO: Build uses ARMv8 advanced-SIMD instruction set.
[/CODE]CPU extensions:

[CODE]m1@599160f8-fb7f-41df-adc2-2b7f4da1aac7 src % sysctl hw.optional
hw.optional.floatingpoint: 1
hw.optional.watchpoint: 4
hw.optional.breakpoint: 6
hw.optional.neon: 1
hw.optional.neon_hpfp: 1
hw.optional.neon_fp16: 1
hw.optional.armv8_1_atomics: 1
hw.optional.armv8_crc32: 1
hw.optional.armv8_2_fhm: 1
hw.optional.armv8_2_sha512: 1
hw.optional.armv8_2_sha3: 1
hw.optional.amx_version: 2
hw.optional.ucnormal_mem: 1
hw.optional.arm64: 1
[/CODE]./Mlucas -s m -cpu 0:7
[CODE]19.1
2048 msec/iter = 3.32 ROE[avg,max] = [0.215347133, 0.312500000] radices = 32 32 32 32 0 0 0 0 0 0
2304 msec/iter = 4.00 ROE[avg,max] = [0.193772149, 0.250000000] radices = 144 32 16 16 0 0 0 0 0 0
2560 msec/iter = 4.28 ROE[avg,max] = [0.178074945, 0.234375000] radices = 160 32 16 16 0 0 0 0 0 0
2816 msec/iter = 4.98 ROE[avg,max] = [0.194841334, 0.281250000] radices = 176 32 16 16 0 0 0 0 0 0
3072 msec/iter = 5.27 ROE[avg,max] = [0.208759866, 0.312500000] radices = 48 32 32 32 0 0 0 0 0 0
3328 msec/iter = 5.94 ROE[avg,max] = [0.324307345, 0.406250000] radices = 208 32 16 16 0 0 0 0 0 0
3584 msec/iter = 6.01 ROE[avg,max] = [0.198822084, 0.250000000] radices = 56 32 32 32 0 0 0 0 0 0
3840 msec/iter = 6.54 ROE[avg,max] = [0.187369624, 0.250000000] radices = 60 32 32 32 0 0 0 0 0 0
4096 msec/iter = 6.88 ROE[avg,max] = [0.176231022, 0.218750000] radices = 64 32 32 32 0 0 0 0 0 0
4608 msec/iter = 7.91 ROE[avg,max] = [0.206297821, 0.281250000] radices = 288 32 16 16 0 0 0 0 0 0
5120 msec/iter = 8.42 ROE[avg,max] = [0.193601628, 0.250000000] radices = 160 16 32 32 0 0 0 0 0 0
5632 msec/iter = 9.70 ROE[avg,max] = [0.221504510, 0.281250000] radices = 352 32 16 16 0 0 0 0 0 0
6144 msec/iter = 10.67 ROE[avg,max] = [0.183728153, 0.250000000] radices = 192 16 32 32 0 0 0 0 0 0
6656 msec/iter = 11.75 ROE[avg,max] = [0.176554163, 0.218750000] radices = 208 16 32 32 0 0 0 0 0 0
7168 msec/iter = 11.84 ROE[avg,max] = [0.213558111, 0.312500000] radices = 224 16 32 32 0 0 0 0 0 0
7680 msec/iter = 13.14 ROE[avg,max] = [0.211455481, 0.281250000] radices = 240 16 32 32 0 0 0 0 0 0
8192 msec/iter = 13.50 ROE[avg,max] = [0.243920143, 0.312500000] radices = 256 16 32 32 0 0 0 0 0 0
9216 msec/iter = 15.44 ROE[avg,max] = [0.256431218, 0.343750000] radices = 288 16 32 32 0 0 0 0 0 0
10240 msec/iter = 16.75 ROE[avg,max] = [0.293991624, 0.375000000] radices = 160 32 32 32 0 0 0 0 0 0
11264 msec/iter = 18.68 ROE[avg,max] = [0.222417407, 0.281250000] radices = 352 16 32 32 0 0 0 0 0 0
12288 msec/iter = 21.39 ROE[avg,max] = [0.219849010, 0.281250000] radices = 192 32 32 32 0 0 0 0 0 0
13312 msec/iter = 23.73 ROE[avg,max] = [0.258116543, 0.312500000] radices = 208 32 32 32 0 0 0 0 0 0
14336 msec/iter = 23.98 ROE[avg,max] = [0.231325382, 0.281250000] radices = 224 32 32 32 0 0 0 0 0 0
15360 msec/iter = 26.76 ROE[avg,max] = [0.235138002, 0.281250000] radices = 240 32 32 32 0 0 0 0 0 0
16384 msec/iter = 26.98 ROE[avg,max] = [0.230396011, 0.312500000] radices = 256 32 32 32 0 0 0 0 0 0
18432 msec/iter = 31.07 ROE[avg,max] = [0.276530284, 0.375000000] radices = 288 32 32 32 0 0 0 0 0 0
20480 msec/iter = 35.91 ROE[avg,max] = [0.229381947, 0.312500000] radices = 320 32 32 32 0 0 0 0 0 0
22528 msec/iter = 37.85 ROE[avg,max] = [0.235262715, 0.296875000] radices = 352 32 32 32 0 0 0 0 0 0
24576 msec/iter = 42.70 ROE[avg,max] = [0.238062530, 0.375000000] radices = 768 16 32 32 0 0 0 0 0 0
26624 msec/iter = 60.50 ROE[avg,max] = [0.254043170, 0.312500000] radices = 208 16 16 16 16 0 0 0 0 0
[/CODE]./Mlucas -s m -cpu 0:3
Looks like threads with heavy load assigned automatically to faster cores.
[CODE]19.1
2048 msec/iter = 3.88 ROE[avg,max] = [0.215133698, 0.312500000] radices = 32 32 32 32 0 0 0 0 0 0
2304 msec/iter = 4.84 ROE[avg,max] = [0.194502305, 0.281250000] radices = 144 32 16 16 0 0 0 0 0 0
2560 msec/iter = 5.00 ROE[avg,max] = [0.184244498, 0.250000000] radices = 40 32 32 32 0 0 0 0 0 0
2816 msec/iter = 6.03 ROE[avg,max] = [0.193770639, 0.250000000] radices = 176 32 16 16 0 0 0 0 0 0
3072 msec/iter = 6.17 ROE[avg,max] = [0.209568299, 0.281250000] radices = 48 32 32 32 0 0 0 0 0 0
3328 msec/iter = 7.15 ROE[avg,max] = [0.221850838, 0.281250000] radices = 52 32 32 32 0 0 0 0 0 0
3584 msec/iter = 7.12 ROE[avg,max] = [0.199199621, 0.281250000] radices = 56 32 32 32 0 0 0 0 0 0
3840 msec/iter = 7.90 ROE[avg,max] = [0.187449630, 0.250000000] radices = 60 32 32 32 0 0 0 0 0 0
4096 msec/iter = 8.21 ROE[avg,max] = [0.174905238, 0.218750000] radices = 64 32 32 32 0 0 0 0 0 0
4608 msec/iter = 9.57 ROE[avg,max] = [0.205330823, 0.281250000] radices = 288 32 16 16 0 0 0 0 0 0
5120 msec/iter = 10.01 ROE[avg,max] = [0.193377434, 0.250000000] radices = 160 16 32 32 0 0 0 0 0 0
5632 msec/iter = 11.74 ROE[avg,max] = [0.221915271, 0.281250000] radices = 352 32 16 16 0 0 0 0 0 0
6144 msec/iter = 12.89 ROE[avg,max] = [0.183260259, 0.250000000] radices = 192 16 32 32 0 0 0 0 0 0
6656 msec/iter = 14.32 ROE[avg,max] = [0.176914974, 0.250000000] radices = 208 16 32 32 0 0 0 0 0 0
7168 msec/iter = 14.40 ROE[avg,max] = [0.213720200, 0.281250000] radices = 224 16 32 32 0 0 0 0 0 0
7680 msec/iter = 16.16 ROE[avg,max] = [0.211763551, 0.281250000] radices = 240 16 32 32 0 0 0 0 0 0[/CODE]
Perfomance looks awesome for mobile CPU. Just to compare timings with AXV-2 on i3-8100 (4 cores): M1 much faster.

AXV-2 on i3-8100:
[CODE]19.1
2048 msec/iter = 4.75 ROE[avg,max] = [0.167383863, 0.218750000] radices = 128 16 16 32 0 0 0 0 0 0
2304 msec/iter = 5.44 ROE[avg,max] = [0.182823637, 0.218750000] radices = 144 16 16 32 0 0 0 0 0 0
2560 msec/iter = 6.29 ROE[avg,max] = [0.224905364, 0.281250000] radices = 160 16 16 32 0 0 0 0 0 0
2816 msec/iter = 6.63 ROE[avg,max] = [0.183906382, 0.230468750] radices = 176 16 16 32 0 0 0 0 0 0
3072 msec/iter = 7.42 ROE[avg,max] = [0.252202803, 0.312500000] radices = 192 16 16 32 0 0 0 0 0 0
3328 msec/iter = 7.52 ROE[avg,max] = [0.225825548, 0.281250000] radices = 208 16 16 32 0 0 0 0 0 0
3584 msec/iter = 8.12 ROE[avg,max] = [0.260567010, 0.375000000] radices = 224 16 16 32 0 0 0 0 0 0
3840 msec/iter = 9.15 ROE[avg,max] = [0.200714048, 0.281250000] radices = 240 16 16 32 0 0 0 0 0 0
4096 msec/iter = 10.92 ROE[avg,max] = [0.165220469, 0.218750000] radices = 64 32 32 32 0 0 0 0 0 0
4608 msec/iter = 11.15 ROE[avg,max] = [0.192892739, 0.250000000] radices = 288 16 16 32 0 0 0 0 0 0
5120 msec/iter = 12.18 ROE[avg,max] = [0.229244523, 0.312500000] radices = 160 32 32 16 0 0 0 0 0 0
5632 msec/iter = 13.47 ROE[avg,max] = [0.187610146, 0.250000000] radices = 352 16 16 32 0 0 0 0 0 0
6144 msec/iter = 16.09 ROE[avg,max] = [0.209471649, 0.281250000] radices = 192 32 32 16 0 0 0 0 0 0
6656 msec/iter = 16.86 ROE[avg,max] = [0.196862667, 0.250000000] radices = 208 16 32 32 0 0 0 0 0 0
7168 msec/iter = 17.38 ROE[avg,max] = [0.196444104, 0.250000000] radices = 224 32 32 16 0 0 0 0 0 0
7680 msec/iter = 23.23 ROE[avg,max] = [0.239954494, 0.343750000] radices = 240 32 32 16 0 0 0 0 0 0
8192 msec/iter = 19.79 ROE[avg,max] = [0.272732764, 0.375000000] radices = 256 32 32 16 0 0 0 0 0 0
9216 msec/iter = 23.01 ROE[avg,max] = [0.242732915, 0.281250000] radices = 288 32 32 16 0 0 0 0 0 0
10240 msec/iter = 27.24 ROE[avg,max] = [0.271287049, 0.375000000] radices = 320 32 32 16 0 0 0 0 0 0
11264 msec/iter = 28.87 ROE[avg,max] = [0.271818621, 0.375000000] radices = 352 32 32 16 0 0 0 0 0 0
12288 msec/iter = 32.04 ROE[avg,max] = [0.259570478, 0.312500000] radices = 768 16 16 32 0 0 0 0 0 0
13312 msec/iter = 37.85 ROE[avg,max] = [0.254703482, 0.312500000] radices = 208 32 32 32 0 0 0 0 0 0
14336 msec/iter = 40.34 ROE[avg,max] = [0.234003331, 0.296875000] radices = 224 32 32 32 0 0 0 0 0 0
15360 msec/iter = 43.84 ROE[avg,max] = [0.245504855, 0.312500000] radices = 960 16 16 32 0 0 0 0 0 0
16384 msec/iter = 45.62 ROE[avg,max] = [0.272600878, 0.375000000] radices = 256 32 32 32 0 0 0 0 0 0
18432 msec/iter = 53.16 ROE[avg,max] = [0.236424995, 0.281250000] radices = 288 32 32 32 0 0 0 0 0 0
20480 msec/iter = 62.92 ROE[avg,max] = [0.237479031, 0.312500000] radices = 320 32 32 32 0 0 0 0 0 0
22528 msec/iter = 66.03 ROE[avg,max] = [0.228240432, 0.312500000] radices = 352 32 32 32 0 0 0 0 0 0
24576 msec/iter = 69.49 ROE[avg,max] = [0.261424145, 0.343750000] radices = 768 16 32 32 0 0 0 0 0 0
[/CODE]
Look forward for their desktop's M1X. Good job. :tu:

ewmayer 2021-02-17 23:02

@Lorenzo:

Thanks for all the timings, that is very useful. I will add a note recommending '-cpu 0:7' for M1 users to the README. Might you have a wall-plug wattmeter you can use to compare (under-load - idle) wattages for those 2 systems, for whatever FFT lengths they are using to run their current GIMPS assignments? I'd be curious to get some idea regarding relative performance-per-watt.

In any event, happy crunching!

Lorenzo 2021-02-18 09:36

[QUOTE=ewmayer;571851]@Lorenzo:

Thanks for all the timings, that is very useful. I will add a note recommending '-cpu 0:7' for M1 users to the README. Might you have a wall-plug wattmeter you can use to compare (under-load - idle) wattages for those 2 systems, for whatever FFT lengths they are using to run their current GIMPS assignments? I'd be curious to get some idea regarding relative performance-per-watt.

In any event, happy crunching![/QUOTE]


Sorry, but I don't have the wall-plug wattmeter.

ldesnogu 2021-02-18 14:32

[QUOTE=Lorenzo;571838]I just want to share my experience with Apple M1 CPU.[/QUOTE]
Thanks for the results!


What machine is that? Mini, MBA or MBP? I'd expect MBA to throttle given the noise my MBP does when running 4 threads :smile:

Lorenzo 2021-02-18 14:46

[QUOTE=ldesnogu;571898]Thanks for the results!


What machine is that? Mini, MBA or MBP? I'd expect MBA to throttle given the noise my MBP does when running 4 threads :smile:[/QUOTE]
Hi. This is an Apple Mac mini M1.

ewmayer 2021-02-18 21:08

[QUOTE=Lorenzo;571901]Hi. This is an Apple Mac mini M1.[/QUOTE]

Some pics via Amazon.com [url=https://www.amazon.com/Apple-Mini-Chip-256GB-Storage/dp/B08N5PHB83]here[/url]. The pic of the rear side shows an exhaust vent similar to those on my Intel NUCs - Lorenzo, are there intake vents on the bottom?

Had a closer look at some of your -cpu 0:7 timings ... the only obvious anomaly is at the very end, 26624K FFT, the timing for that is anomalously large. This is mainly for down-the-road as this FFT is way beyond the GIMPS wavefront, but looking at the pattern of best-timing FFT radices for the rows above it, this machine seems to really like larger leading FFT radices (call the leftmost radix r0) and combos of the form r0,16,32,32 and r0,32,32,32. At this 26M FFT length, there is no such available combo because I did not (yet) implement a radix-416 FFT-pass routine, thus instead of 416,32,32,32 the best we can do is 208,16,16,16,16, which means an extra pass through the data each iteration.

If you wold be so kind, could you pause any running jobs (I believe 'kill -STOP [pid]' works on MacOS same as Linux, then 'kill -CONT [pid]' to resume, and either 'pidof' or 'top' will give you the process ID), and re-run just the 26M-FFT timing? Here is how:
[i]
./Mlucas -iters 1000 -cpu 0:7 -fftlen 26624 >& test.log
[/i]
After that completes, paste the new last-line that got appended to mlucas.cfg as a result, and please attach the test.log . Thanks.

Lorenzo 2021-02-18 22:12

1 Attachment(s)
[QUOTE=ewmayer;571948]... the only obvious anomaly is at the very end, 26624K FFT, the timing for that is anomalously large.[/QUOTE]
Actually when I posted results for i-8100 I did cut the line with timings for 26624. I thought the same, that it was some heavy load from background application when I did the benchmark.
So I just tried to make redoing on i-8100 (OS Oracle Linux 7) and I see the same: big jump from ~69 to ~141 msec exactly for 26624.
So I think it's not a platform specific issue.
[CODE] 18432 msec/iter = 53.16 ROE[avg,max] = [0.236424995, 0.281250000] radices = 288 32 32 32 0 0 0 0 0 0
20480 msec/iter = 62.92 ROE[avg,max] = [0.237479031, 0.312500000] radices = 320 32 32 32 0 0 0 0 0 0
22528 msec/iter = 66.03 ROE[avg,max] = [0.228240432, 0.312500000] radices = 352 32 32 32 0 0 0 0 0 0
24576 msec/iter = 69.49 ROE[avg,max] = [0.261424145, 0.343750000] radices = 768 16 32 32 0 0 0 0 0 0
26624 msec/iter = 144.86 ROE[avg,max] = [0.272725339, 0.343750000] radices = 52 16 16 32 32 0 0 0 0 0
26624 msec/iter = 141.33 ROE[avg,max] = [0.272368315, 0.375000000] radices = 52 16 16 32 32 0 0 0 0 0
24576 msec/iter = 68.38 ROE[avg,max] = [0.261777142, 0.359375000] radices = 768 16 32 32 0 0 0 0 0 0
26624 msec/iter = 141.06 ROE[avg,max] = [0.272368315, 0.375000000] radices = 52 16 16 32 32 0 0 0 0 0[/CODE]

Lorenzo 2021-02-19 07:45

Full test for large fft on i3-8100:
[CODE] 8192 msec/iter = 19.61 ROE[avg,max] = [0.272732764, 0.375000000] radices = 256 32 32 16 0 0 0 0 0 0
9216 msec/iter = 23.07 ROE[avg,max] = [0.239072536, 0.312500000] radices = 288 16 32 32 0 0 0 0 0 0
10240 msec/iter = 27.33 ROE[avg,max] = [0.271287049, 0.375000000] radices = 320 32 32 16 0 0 0 0 0 0
11264 msec/iter = 28.73 ROE[avg,max] = [0.271818621, 0.375000000] radices = 352 32 32 16 0 0 0 0 0 0
12288 msec/iter = 32.18 ROE[avg,max] = [0.259570478, 0.312500000] radices = 768 16 16 32 0 0 0 0 0 0
13312 msec/iter = 36.87 ROE[avg,max] = [0.254703482, 0.312500000] radices = 208 32 32 32 0 0 0 0 0 0
14336 msec/iter = 39.92 ROE[avg,max] = [0.234003331, 0.296875000] radices = 224 32 32 32 0 0 0 0 0 0
15360 msec/iter = 42.65 ROE[avg,max] = [0.245504855, 0.312500000] radices = 960 16 16 32 0 0 0 0 0 0
16384 msec/iter = 44.85 ROE[avg,max] = [0.272600878, 0.375000000] radices = 256 32 32 32 0 0 0 0 0 0
18432 msec/iter = 52.67 ROE[avg,max] = [0.236424995, 0.281250000] radices = 288 32 32 32 0 0 0 0 0 0
20480 msec/iter = 61.48 ROE[avg,max] = [0.237479031, 0.312500000] radices = 320 32 32 32 0 0 0 0 0 0
22528 msec/iter = 65.70 ROE[avg,max] = [0.228240432, 0.312500000] radices = 352 32 32 32 0 0 0 0 0 0
24576 msec/iter = 68.40 ROE[avg,max] = [0.261424145, 0.343750000] radices = 768 16 32 32 0 0 0 0 0 0
26624 msec/iter = 141.14 ROE[avg,max] = [0.272725339, 0.343750000] radices = 52 16 16 32 32 0 0 0 0 0
28672 msec/iter = 106.92 ROE[avg,max] = [0.252042892, 0.312500000] radices = 224 16 16 16 16 0 0 0 0 0
30720 msec/iter = 114.56 ROE[avg,max] = [0.288327813, 0.375000000] radices = 240 16 16 16 16 0 0 0 0 0
32768 msec/iter = 101.20 ROE[avg,max] = [0.238132941, 0.312500000] radices = 1024 16 32 32 0 0 0 0 0 0
36864 msec/iter = 137.73 ROE[avg,max] = [0.265349020, 0.312500000] radices = 288 16 16 16 16 0 0 0 0 0
40960 msec/iter = 161.66 ROE[avg,max] = [0.251543120, 0.312500000] radices = 320 16 16 16 16 0 0 0 0 0
45056 msec/iter = 170.85 ROE[avg,max] = [0.244248223, 0.312500000] radices = 352 16 16 16 16 0 0 0 0 0
49152 msec/iter = 153.04 ROE[avg,max] = [0.255821747, 0.343750000] radices = 768 32 32 32 0 0 0 0 0 0
53248 msec/iter = 293.04 ROE[avg,max] = [0.262757669, 0.312500000] radices = 52 16 32 32 32 0 0 0 0 0
57344 msec/iter = 270.81 ROE[avg,max] = [0.265370288, 0.375000000] radices = 224 16 16 16 32 0 0 0 0 0
61440 msec/iter = 204.05 ROE[avg,max] = [0.246525841, 0.343750000] radices = 960 32 32 32 0 0 0 0 0 0[/CODE]

LaurV 2021-02-19 09:20

1 Attachment(s)
[QUOTE=Lorenzo;571883]Sorry, but I don't have the wall-plug wattmeter.[/QUOTE]
Whaaaaattttt? :shock:
You must buy one, try Aliexpress, [URL="https://www.aliexpress.com/item/1005001462804114.html"]here[/URL], you can even smart-measure some parameters of your "wife" with it! (whatever that means :razz:)
[ATTACH]24350[/ATTACH]
(photo for posterity, in case they change it; to be clear, this is a joke, I do not promote nor endorse that product, but "one click operation for wife" I would buy any time!).

ewmayer 2021-02-19 19:53

[QUOTE=Lorenzo;571993]Full test for large fft on i3-8100:
[snip][/QUOTE]

Thanks - that is very helpful as far as future roadmapping goes - so for selected of the FFT lengths:
o 26M: r0 = 208 needs to be made more accurate (rejected in your test.log due to excess ROE), also need r0 = 416;
o 28,30M: Need r0 = 448,480;
o 36,40,44,52,56M: Need r0 = 576,640,704,832,896.

bayanne 2021-05-10 13:12

Apple M1 on iMac
 
My new iMac will be arriving shortly, and I hope to compile and run MLucas v19.1 on it.

However where do I find the simple version on how to compile on ARM cpu?

Uncwilly 2021-05-10 14:03

[QUOTE=LaurV;571998]Whaaaaattttt? :shock:
You must buy one, try Aliexpress,[/QUOTE]
Aliexpress wants an image of the front of my government ID. That is a hard nope. If it was located in my country that would be a nope. For a company behind the great firewall it is NOPE[SIZE="4"][SUP]∞[/SUP][/SIZE]

kriesel 2021-05-10 15:10

aFor about half the stated price, and no ID theft cooperation required
[URL]https://www.ebay.com/itm/301323374960[/URL]
and plugs into a standard US 120VAC 15A or 20A outlet. Variations are available for other nations' standards.
All these, including the original [URL="https://www.globalindustrial.com/p/energy-saver?infoParam.campaignId=WR&msclkid=77b176673ab21083db9116fe748ca608&utm_source=bing&utm_medium=cpc&utm_campaign=%5BADL%5D%20%5BPLA%5D%20%5BTools%20%26%20Instruments%5D%20%7BTest,%20Measurement%20%26%20Inspection%7D%20(Neutral)%20-%20%5BNormal%20SKUs%5D&utm_term=4582489600871789&utm_content=Tools%20%26%20Instruments%20-%20Test,%20Measurement%20%26%20Inspection%20-%20(Neutral)&adlclid=77b176673ab21083db9116fe748ca608"]KillaWatt,[/URL] obstruct both outlets of a duplex outlet.
Your local building supply or hardware store will gladly supply a cube tap or multi-outlet power strip to deal with that.

ewmayer 2021-05-10 21:10

[QUOTE=bayanne;578137]My new iMac will be arriving shortly, and I hope to compile and run MLucas v19.1 on it.

However where do I find the simple version on how to compile on ARM cpu?[/QUOTE]

Hi, Bayanne - tdulcet and danc2 have put together a very nice auto-install-and-tune script, but that is alas Linux-only. But a simple DIY command-line build for Arm-based iMac is straightforward, here the condensed instructions - this assumes you have the native Clang compiler installed, 'which clang' to check that. First download the v19.1 source tarball, which is linked at the Mlucas [url=http://www.mersenneforum.org/mayer/README.html]README[/url] page, unpack as detailed, then
[i]
cd ~/*19.1
mkdir obj
cd obj
clang -c -O3 -DUSE_ARM_V8_SIMD -DUSE_THREADS ../src/*.c >& build.log
grep error build.log
[Assuming above grep comes up empty]clang -o Mlucas *.o -lm -lpthread -lrt
[/i]
Note Clang does not need explicit library-linkage of the math, pthread and realtime libraries, i.e. does not need the user to invoke [i]-lm -lpthread -lrt[/i] at link time; if you include them and get a Clang linker error, try omitting them and relinking with just [i]clang -o Mlucas *.o[/i]

To autotune for 8-core iMac M1, a hybrid of 4 'performance' and 4 'efficiency' cores: [i]./Mlucas -s m -iters 1000 -cpu 0:7[/i]

Assuming all that works, please post a copy of the resulting mlucas.cfg file containing optimal-FFT-parameters for your machine here. You can do assignments auto-management using the primenet.py script, a copy of which is in ~/*19.1/src . I suggest you create a run-directory at top level in your user-dir, copy the Mlucas binary, mlucas.cfg and primenet.py there, then run primenet.py as described at the README.

bayanne 2021-05-11 09:16

Thanks, will let you know how I get on ...

ewmayer 2021-05-11 20:10

How much RAM are you getting in your iMac? I really dislike the lack of RAM-expandability in the M1-based Apple offerings, as detailed in articles like this:

[url=https://appleinsider.com/articles/21/04/06/theoretically-you-can-upgrade-ram-ssd-on-your-m1-mac-mini-but-you-shouldnt]Theoretically, you can upgrade RAM & SSD on your M1 Mac mini, but you shouldn't[/url] | AppleInsider

Been busy last few months adding p-1 factoring support to the in-development v20 code - stage 2 really benefits from lots of RAM. Glad I always use humbler micro-PC-style systems among my optimization targets, making sure stage 2 performs at least halfway-decently with just 4-16GB RAM will also come in handy on M1 systems.

bayanne 2021-05-22 06:31

[QUOTE]How much RAM are you getting in your iMac?[/QUOTE]

As much as I was able to get, i.e. 16gb

ewmayer 2021-05-22 21:27

[QUOTE=bayanne;578847]As much as I was able to get, i.e. 16gb[/QUOTE]

For p-1 of current-wavefront (6M FFT) exponents, that should allow you to cut 20-25% work off the minimum-memory option for my stage 2 implementation, depending on how much of it you allow stage 2 to use: 8GB => 20%, 12GB => 25%. I'm working on the v20 stage 2 restart-from-interrupt code right now.

bayanne 2021-06-12 04:29

[QUOTE=ewmayer;578883]For p-1 of current-wavefront (6M FFT) exponents, that should allow you to cut 20-25% work off the minimum-memory option for my stage 2 implementation, depending on how much of it you allow stage 2 to use: 8GB => 20%, 12GB => 25%. I'm working on the v20 stage 2 restart-from-interrupt code right now.[/QUOTE]

Stuck at present, sent you a PM ...

bayanne 2021-06-18 05:35

[QUOTE=bayanne;580749]Stuck at present, sent you a PM ...[/QUOTE]

Now built, and crunching away, thanks for your help.

Is there a way of contacting Primenet with a progress report on the exponent?

ewmayer 2021-06-18 19:57

[QUOTE=bayanne;581310]Is there a way of contacting Primenet with a progress report on the exponent?[/QUOTE]

As I noted in my earlier e-mail, see the "Get exponents from Primenet" section of the README - it's linked in the Index at top of the page; please read it carefully, as it's your first go.

You'll need to copy the primenet.py script from your /src dir to whatever dir(s) you are planning to run instances from - for Apple M1, just a single instance using 1-8 threads (depending on how much you want to stress your CPU & cooling) seems a good compromise between throughput and ease-of-use.

bayanne 2021-06-19 11:09

[QUOTE=ewmayer;581365]As I noted in my earlier e-mail, see the "Get exponents from Primenet" section of the README - it's linked in the Index at top of the page; please read it carefully, as it's your first go.

You'll need to copy the primenet.py script from your /src dir to whatever dir(s) you are planning to run instances from - for Apple M1, just a single instance using 1-8 threads (depending on how much you want to stress your CPU & cooling) seems a good compromise between throughput and ease-of-use.[/QUOTE]

I probably did not express my request very clearly. I was hoping to update my Account Assignments Details which at present only show that my assignment was assigned on 2021-06-16 and that the last update was the same day. I have tried running primenet.py but it has not updated my progress on the exponent, or given the the estimated finish time.
Can this be done with primenet.py?

M344587487 2021-06-19 12:55

[QUOTE=bayanne;581310]Now built, and crunching away, thanks for your help.

Is there a way of contacting Primenet with a progress report on the exponent?[/QUOTE]
If you have a watt meter, it'd be interesting to know how much power the M1 chip draws under mlucas load. I've only just noticed Lorenzo's stats otherwise I'd be bugging you for that too, what a tasty thing to have missed.


Apple's chips are intriguing, almost to the point where they may break down my incredibly anti-Apple stance. In a year or two if a Linux port gets stable enough that mac OS can comfortably be nuked from orbit they may get some cash from me yet.

ewmayer 2021-06-19 20:37

[QUOTE=bayanne;581393]I probably did not express my request very clearly. I was hoping to update my Account Assignments Details which at present only show that my assignment was assigned on 2021-06-16 and that the last update was the same day. I have tried running primenet.py but it has not updated my progress on the exponent, or given the the estimated finish time.
Can this be done with primenet.py?[/QUOTE]

I refer you again to the README:

"You must be connected to the internet when you launch the script; once it has done its initial work-fetching you can be offline most of the time; the program will simply periodically check whether there are any new results in the run directory in which it was launched; if yes *and* it is able to connect to the PrimeNet server it will submit the new results (usually just one, unless you are offline nearly all the time) and fetch new work; otherwise it will sleep and retry later. [b]The default is to check for 'results to submit/work to get?' every 6 hours[/b]; you may override this via the -t option, followed by your desired time interval in seconds. '-t 0' means run a single-shot get-work-to-do and quit, if for some reason you prefer to periodically run the script manually yourself."

That periodic checkin also updates assignment details - my 'cpus' page is up to date.

1. What *exact* command did you use to run the py-script, and what was the resulting terminal output?

2. Does 'ps aux |grep primenet' show a running primenet.py?

3. If 'yes' to [2], does the exponent page show any update in last 12 hours?

bayanne 2021-06-20 05:00

[QUOTE=ewmayer;581411]I refer you again to the README:

"You must be connected to the internet when you launch the script; once it has done its initial work-fetching you can be offline most of the time; the program will simply periodically check whether there are any new results in the run directory in which it was launched; if yes *and* it is able to connect to the PrimeNet server it will submit the new results (usually just one, unless you are offline nearly all the time) and fetch new work; otherwise it will sleep and retry later. [b]The default is to check for 'results to submit/work to get?' every 6 hours[/b]; you may override this via the -t option, followed by your desired time interval in seconds. '-t 0' means run a single-shot get-work-to-do and quit, if for some reason you prefer to periodically run the script manually yourself."

That periodic checkin also updates assignment details - my 'cpus' page is up to date.

1. What *exact* command did you use to run the py-script, and what was the resulting terminal output?

2. Does 'ps aux |grep primenet' show a running primenet.py?

3. If 'yes' to [2], does the exponent page show any update in last 12 hours?[/QUOTE]

I used the command :
python primenet.py -d -T 100 -u [Username] -p [password]

I entered ps aux lgrep primenet
and received:

ps: illegal argument: lgrep
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
[-u]
[-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
ps [-L]

I then restarted python primenet.py
and received:

primenet.py: send_progress: Cannot update, the registration is not done
primenet.py: send_progress: Call primenet.py with --register option

I then entered ps aux lgrep primenet
and nothing

Please forgive my lack of familiarity with these commands and instructions

bayanne 2021-06-20 05:01

[QUOTE=M344587487;581395]If you have a watt meter, it'd be interesting to know how much power the M1 chip draws under mlucas load. I've only just noticed Lorenzo's stats otherwise I'd be bugging you for that too, what a tasty thing to have missed.
[/QUOTE]

Sorry, I don't have one

Happy5214 2021-06-20 06:30

[QUOTE=ewmayer;581411]2. Does 'ps aux |grep primenet' show a running primenet.py?[/QUOTE]

[QUOTE=bayanne;581428]I entered ps aux lgrep primenet
and received:

ps: illegal argument: lgrep
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
[-u]
[-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
ps [-L]
[/QUOTE]

Just a drive-by assist here. Spacing and a bad font choice might have been problems in ewmayer's post there. The invocation was [c]ps aux | grep primenet[/c], with a vertical bar between [c]aux[/c] and [c]grep[/c]. (This pipes, or redirects, the output of [C]ps aux[/C] to [C]grep primenet[/C], which will search for the string [C]"primenet"[/C] in the process list it's receiving.) bayanne accidentally used a lowercase L instead of a bar character, which caused the illegal argument error above.

bayanne 2021-06-20 11:15

[QUOTE=Happy5214;581433]Just a drive-by assist here. Spacing and a bad font choice might have been problems in ewmayer's post there. The invocation was [c]ps aux | grep primenet[/c], with a vertical bar between [c]aux[/c] and [c]grep[/c]. (This pipes, or redirects, the output of [C]ps aux[/C] to [C]grep primenet[/C], which will search for the string [C]"primenet"[/C] in the process list it's receiving.) bayanne accidentally used a lowercase L instead of a bar character, which caused the illegal argument error above.[/QUOTE]

Thanks, tried ps aux | grep primenet
but still nothing

ewmayer 2021-06-20 20:54

[QUOTE=bayanne;581428]primenet.py: send_progress: Cannot update, the registration is not done
primenet.py: send_progress: Call primenet.py with --register option[/QUOTE]

That 2nd error line is telling you what to do - the first time you invoke the now-Primenet-API-enabled script, you need to do some basic registration via 'primenet.py with --register'. Try that, and see what it prompts you for. Once done, reinvoke the script using your original command sequence.

That's on me - I see I forgot to mention the --register step on the README. Go ahead and try it, and I'll use your result/feedback to update the README.

bayanne 2021-06-21 09:50

[QUOTE=ewmayer;581462]That 2nd error line is telling you what to do - the first time you invoke the now-Primenet-API-enabled script, you need to do some basic registration via 'primenet.py with --register'. Try that, and see what it prompts you for. Once done, reinvoke the script using your original command sequence.

That's on me - I see I forgot to mention the --register step on the README. Go ahead and try it, and I'll use your result/feedback to update the README.[/QUOTE]

My computer is registered, but still no further forward about being able to provide updates to Primenet

tdulcet 2021-06-21 15:36

[QUOTE=ewmayer;581462]That 2nd error line is telling you what to do - the first time you invoke the now-Primenet-API-enabled script, you need to do some basic registration via 'primenet.py with --register'. Try that, and see what it prompts you for. Once done, reinvoke the script using your original command sequence.

That's on me - I see I forgot to mention the --register step on the README.[/QUOTE]

Just to note, Daniel and I's updated/[URL="https://www.mersenneforum.org/showthread.php?t=26574"][FONT=Arial]improved [/FONT]PrimeNet script[/URL] automatically registers the computer with PrimeNet if the user does NOT provide a password. The password is not needed because it can use the PrimeNet API to get assignments, just as Prime95/MPrime does.

For reference, the first time it can be run like this to just register the instance with PrimeNet:
[C]python3 primenet.py -d -t 0 -T <Worktype> -u <PrimeNet User ID>[/C]

and then it can subsequently be run like this to automatically get assignments, report assignment results and progress to PrimeNet:
[C]python3 primenet.py -d[/C]

If setting up more than one worker/run on the computer, the the above two commands are:
[C]python3 primenet.py -d -t 0 -T <Worktype> -u <PrimeNet User ID> --num_workers <Number of workers>[/C]
and
[C]python3 primenet.py -d -c <Worker/CPU core number>[/C]

See [URL="https://github.com/tdulcet/Distributed-Computing-Scripts#primenet"]here[/URL] or run [C]python3 primenet.py --help[/C] for the full list of supported options. The [URL="https://www.mersenneforum.org/mayer/README.html#download1"]Mlucas install script[/URL] can automate all of this for Linux users.

ewmayer 2021-06-21 19:53

[QUOTE=bayanne;581477]My computer is registered, but still no further forward about being able to provide updates to Primenet[/QUOTE]

Meaning you tried 'python primenet.py --register'? What happened on doing that? That should prompt for your primenet login userid. I just tried it from a newly-created test directory on one of my boxes:
[i]
[ewmayer@localhost obj_avx512]$ python primenet.py --register
Usage: primenet.py [options]

primenet.py: error: To register the instance, --username and --hostname are required
[/i]
OK, retry with
[i]
python primenet.py --register --username ewmayer --hostname knl_test
GUID 9e4336ce6f928e63d0f19696f688720f correctly registered with the following features:
Username: ewmayer
Hostname: knl_test
CPU model: cpu.unknown
CPU features:
CPU L1 cache size: 8kB
CPU L2 cache size: 512kB
CPU cores: 1
CPU thread per core: 0
CPU frequency: 100MHz
Memory size: 0MB
If you want to change the value, please rerun with the corresponding options or edit the local.ini file and rerun with --register option
You can see the result in this page:
[url]https://www.mersenne.org/editcpu/?g=9e4336ce6f928e63d0f19696f688720f[/url]
[/i]
Registration creates a local.ini file, here mine from the above:
[code][primenet]
worktype = 101
num_cache = 1
percent_limit = 90
hostname = knl_test
cpu_model = cpu.unknown
features =
frequency = 100
memory = 0
l1 = 8
l2 = 512
np = 1
hp = 0
username = ewmayer
guid = 9e4336ce6f928e63d0f19696f688720f[/code]
Then reinvoke the script using the syntax from the README:
[i]
python primenet.py -d [-T [worktype]] -u [uid] -p [pwd] [-t [frequency]] &
[/i]
Note since registration has created a default worktype and most folks are OK with the default every-6-hour updates, so you really now just want
[i]
python primenet.py -d -u [uid] -p [pwd]
[/i]
That will add couple fields to local.ini after the first few updates-to-server, including

password = [pwd]
usec_per_iter = [last per-iter-time for current assignment]

======================

@tdulcet: Correct me if I'm wrong, but I thought your setup script is Linux-only. Bayanne is using a Mac.

bayanne 2021-06-22 04:53

OK, sorted now
used:

python primenet.py -d

This created estimated completion, and days to go
Used:

python primenet.py --register --username[] --hostname[]

And this confirmed registration details

Thanks to you both.

tdulcet 2021-06-22 11:47

[QUOTE=ewmayer;581505]@tdulcet: Correct me if I'm wrong, but I thought your setup script is Linux-only. Bayanne is using a Mac.[/QUOTE]

Yes, my setup/install script is Linux only, but our PrimeNet script will work on Windows, macOS and Linux, the same as Loïc Le Loarer's PrimeNet script, which ours was built on. The commands I listed above were for manually using the PrimeNet script independently of the Mlucas install script.

The commands and output for our PrimeNet script is roughly the same as what you posted, except there is no [C]-r/--register[/C] option since that is automatic. With Loïc's PrimeNet script, the [C]-r/--register[/C] option controlled whether it either used the PrimeNet API to report assignment results and progress or only did manual testing. With Daniel and I's PrimeNet script, the [C]-p/--password[/C] option controls this instead. If the user provides a password, it will only do manual testing and will not report the progress (this was of course the default behavior for old versions of the PrimeNet script before Loïc Le Loarer's version). If they do NOT provide a password, it will completely use the PrimeNet v5 API, just as Prime95/MPrime does.

There are other minor differences too, including different defaults. That [C]usec_per_iter[/C] field was renamed to [C]msec_per_iter[/C], since the value of course is in milliseconds not microseconds. It also now defaults to hourly updates to support PRP proof uploading with GpuOwl and eventually Mlucas v20.1 (the Linux install script will keep the default 6 hour updates until v20.1 is released). Please see the [URL="https://www.mersenneforum.org/showthread.php?t=26574"]dedicated thread[/URL] for a long list of all our changes/improvements.

ewmayer 2021-07-05 20:19

[b]Bug Alert:[/b] This bug will cause around 6% of completed results to not be successfully submitted to the server via the v5-APIified primenet.py script.

I first discovered the issue in March, around a month after v19.1 came out. It affects both v19.0 and 19.1. After reporting the problem to George and James Heinrich, I mis-read James' reply as meaning he had patched the API to accept the results in question; turns out he only patched the Manual Submission page. It seems there is no good way to work around this on the server/API-side, so users reading this, please check your nohup.out files for API error messages (keywords bolded) of the kind listed below, and then manually edit your results.txt file(s) to fix the offending entries.
[i]
primenet.py: submit_one_line_v5: Submitting using V5 API
{"status":"C", "exponent":63327331, "worktype":"LL", "res64":" 5DFBB5BD815400A", "fft-length":3670016, "shift-count":3025624, "error-code":"00000000", "program":{"name":"Mlucas", "version":"19.1"}, "timestamp":"2021-07-03 17:47:01 UTC", "aid":"7D690A965943BDCE33448D5072CF9AE1"}
primenet.py: submit_one_line_v5: ERROR while submitting result on mersenne.org: assignment_id=7D690A965943BDCE33448D5072CF9AE1
primenet.py: submit_one_line_v5: INVALID PARAMETER: this is a bug in primenet.py, please notify the author
primenet.py: submit_one_line_v5: Reason: parameter rd: [b]Invalid value/precision[/b]: ' 5DFBB5BD815400A'
[/i]
The problem is due to a silly print-leading-zeros formatting bug in the new JSON-result-format function I added in v19 - for the res64-field I used C print format %16llX instead of %016llX - the former causes any leading 0-hexits in the res64 to get printed as blanks, and the server treats such less-than-16-hexits-wide results as 'Invalid'.

The problem has been fixed in the Mlucas v20 dev-code (which I plan to release by end of this week) - the reason I didn't rush out a patch to v19 is 2-fold:

1 - I thought James had successfully worked around the problem via API-side hack;

2 - Patched code only solves the problem going forward - I didn't want to ask all the users to manually scan their results files for completed runs left in submission limbo doe to this.

In your run directories, assuming you invoke Mlucas v19 in the recommended way, via 'nohup ./Mlucas [whatever]', do 'grep Invalid nohup.out'. If that shows 1 or more matches, edit results.txt and fill the leading blanks in the corresponding res64 fields with 0s, then save the file. The primenet.py script will automatically try resubmitting the affected results on its next every-6-hour autoexec; I suggest you wait that long, then at your next opportunity check any such assignments for successful submission 2 ways:

1. grep for the exponent or res64 in results_sent.txt;

2. In a browser, load [url]https://www.mersenne.org/report_exponent/?exp_lo=[/url][your exponent here]&full=1 .

Alas, due to the elapsed months since I first noticed/reported the issue and having the not-patched-on-API-side-ness brought to my attention over the weekend, some of the affected results will inevitably have expired and been reassigned. For that, I apologize.

tdulcet 2021-07-06 10:15

[QUOTE=ewmayer;582640]
In your run directories, assuming you invoke Mlucas v19 in the recommended way, via 'nohup ./Mlucas [whatever]', do 'grep Invalid nohup.out'. If that shows 1 or more matches, edit results.txt and fill the leading blanks in the corresponding res64 fields with 0s, then save the file. The primenet.py script will automatically try resubmitting the affected results on its next every-6-hour autoexec;[/QUOTE]

I just updated our PrimeNet script so it will automatically handle this without users needing to manually edit their [C]results.txt[/C] file. Users can just [URL="https://github.com/tdulcet/Distributed-Computing-Scripts/raw/master/primenet.py"]download[/URL] the new version over their existing copy.

Users of Loïc Le Loarer's older PrimeNet script (the one included with Mlucas v19.1) could instead make the same changes I made to our improved version: Replace the [URL="https://github.com/llloic11/primenet/blob/main/primenet.py#L625-L633"]two occurrences[/URL] of [C]ar['res64'][/C] with [C]ar['res64'].strip().zfill(16)[/C]. It will then automatically remove any blanks from the residue and add the needed leading zeros before submitting results to the PrimeNet server.

ewmayer 2021-07-06 19:40

[QUOTE=tdulcet;582680]I just updated our PrimeNet script so it will automatically handle this without users needing to manually edit their [C]results.txt[/C] file. Users can just [URL="https://github.com/tdulcet/Distributed-Computing-Scripts/raw/master/primenet.py"]download[/URL] the new version over their existing copy.

Users of Loïc Le Loarer's older PrimeNet script (the one included with Mlucas v19.1) could instead make the same changes I made to our improved version: Replace the [URL="https://github.com/llloic11/primenet/blob/main/primenet.py#L625-L633"]two occurrences[/URL] of [C]ar['res64'][/C] with [C]ar['res64'].strip().zfill(16)[/C]. It will then automatically remove any blanks from the residue and add the needed leading zeros before submitting results to the PrimeNet server.[/QUOTE]

Thanks, Teal - that's a very useful fiddle, I will post a link to the so-hacked Loarer-script in a v19 patch alert to the README. It should be moot in a few weeks when v20 is up, but for folks not reading this who are running v19 and then see the v20 thread and build that, including the modified script with that source base will automatically patch up any v19 results they have which are in limbo due to the print-bug, assuming they use the same rundir(s) as before and start v20 on any incomplete v19 runs they have.


All times are UTC. The time now is 04:50.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.