mersenneforum.org > YAFU Featured request
 Register FAQ Search Today's Posts Mark Forums Read

2016-04-04, 18:41   #210
Sergei Chernykh

Jun 2015
Stockholm, Sweden

8310 Posts

Quote:
 Originally Posted by bsquared I've verified this works too, and removed the single-thread limitation if linked against ECM 7+. It is unlikely there will be another official release. The best I can do anymore is fix bugs and occasionally work on something interesting that hopefully doesn't break something else. If you can't compile from SVN for your platform, chances are someone can and would be willing to help out.
I can compile it, but the problem is that the resulting binary is a bit slower than the official yafu 1.34. I can't get proper compiler options to make it at least as fast.

2016-04-04, 20:12   #211
bsquared

"Ben"
Feb 2007

63148 Posts

Quote:
 Originally Posted by Sergei Chernykh I can compile it, but the problem is that the resulting binary is a bit slower than the official yafu 1.34. I can't get proper compiler options to make it at least as fast.
I assume you mean SIQS is slower? The speed of ECM and NFS will be dictated by builds of gmp, gmp-ecm, msieve, and ggnfs.

There are only a couple things you can do to impact SIQS speed. The first and least important is the march flag, which you can change on line 19 of the makefile. The second and the most important are the following arguments to the make process:

To add sse41 assembly optimizations make like this
Code:
make x86_64 NFS=1 USE_SSE41=1
Or if you have a haswell system or above you can now do
Code:
make x86_64 NFS=1 USE_AVX2=1
which is 10-15% faster than sse41.

The last is the compiler. Either of the above works with gcc. If you have the Intel compiler it will make everything a little bit faster yet. (COMPILER=icc during make, if you have it)

 2016-04-05, 01:58 #212 Mr. Odd   Mar 2010 53 Posts As ever, if someone is willing to compile it for Ubuntu/Linux, that'd be greatly appreciated.
2016-08-21, 21:27   #213
Joe O

Aug 2002

3·52·7 Posts

Quote:
 Originally Posted by richs Is it possible to write the status of the ECM effort to the factor.log file when one interrupts an ECM run with Control-C, for example: t15: 78.14 t20: 36.48 t25: 5.09 t30: 0.54 t35: 0.04 sum of completed work is t27.68 Thanks!
Has this been implemented? Can it be? It would be a great help for me, and possibly others that have to share machines.

2016-08-24, 13:57   #214
bsquared

"Ben"
Feb 2007

CCC16 Posts

Quote:
 Originally Posted by Joe O Has this been implemented? Can it be? It would be a great help for me, and possibly others that have to share machines.
This has now been implemented; latest code is in \braches\wip in SVN 358.

2016-08-25, 11:44   #215
Joe O

Aug 2002

52510 Posts

Quote:
 Originally Posted by bsquared This has now been implemented; latest code is in \braches\wip in SVN 358.
Thank You!

 2016-11-08, 03:33 #216 LaurV Romulan Interpreter     Jun 2011 Thailand 5×11×157 Posts Arrr.. Huston? We have a problem... I remember long arguments about packed versus unpacked relations, but I don't remember the result. I may have been the only one arguing for unpacked relations, that's why. Was there a command line switch implemented like "-unpack" or so? I generally avoid using the newest versions of yafu which had the "bad habit" of packing my beautiful relations, and as I argued at the time of the discussion, I see no benefit in packing them. Today's disks (ramdisk, ssd) are so fast that it is no benefit in having smaller file, and there is no speedup of one version to another, and if you have 1 gig on the disk for the packed file, then you also have 3 gigs for the unpacked one... Most factorizations we (general we) do anyhow (in the 120-130 digits), need only a tenth of that. If you are concerned about sending the relation files to third party, guess what, a "dedicated" external packer as **zip or **rar can do wonders, and compress better and faster than internal yafu's lzw, not counting the crc-ing or encrypting etc. the file for a better and safer transfer. On the other side, unpacked relations file is "safer", if the file gets corrupted, just open it with pn2 or your favorite text editor and delete the corrupted relations, but you can save the most of the other and don't need to re-do all the days/weeks of sieving. In a packed file, corruption usually means discarding all the file, or at best, the lzw block where the byte was corrupted. And from the "luck" point of view, at a size of gigabytes and a lot of writing, and ramdisks mounting and unmounting, there is a high chance that will happen sooner or later. Therefore I was quite happy with my "unpacking" yafu. Till today. Some time ago I installed a 32-bit yafu in a win32 computer (no, it can not be upgraded, due to other tasks running there, I already answered to this, thanks!) which was crunching some aliquots in its 4 stupid and slow cores. Yesterday I realized that there is no progress there for the last week, and moved my buttons to go there and have a look. My toy was crunching the C150 from aliquot 714192 index 2420, and it already tried ~20 times to go into LA, crashing every time at memory allocation. That computer only has 4 gigs of RAM, from which win32 only uses 3, so the crash was normal. Code: found 3777669 cycles, need 3759956 weight of 3759956 cycles is about 263367037 (70.05/cycle) .... commencing cycle optimization start with 20429912 relations pruned 382710 relations memory use: 563.3 MB distribution of cycle lengths: 1 relations: 537300 2 relations: 485096 .... 9 relations: 169164 10+ relations: 552040 heaviest cycle: 22 relations RelProcTime: 1374 nfs: commencing msieve linear algebra commencing linear algebra read 3759956 cycles cycles contain 12204676 unique relations failed to allocate 292912224 bytes WARNING: couldn't find factor in factor.log And "da capo al fine", starts again with filtering. For ever. Grrrr.. Here is aliqueit.exe's fault, it should give a warning and stop, or switch to other sequence, not call yafu forever. But that is another story. I decided to move the nfs.dat with all afferent job files to another computer, running yafu in 64 bits and having reasonable amount of RAM. And fucking surprize! The nfs.dat is compressed. I had no idea that I installed one "bad yafu" in that computer, all my other installations are "good yafu" which do not compress the relation file. The ugly part is that new yafu with "uncompressing" does not realize that the .dat file is compressed, and when you resume with it, it will stay 20 minutes (the compressed file has 2.9 GB) looking for the "special q", of course it does not find any, as it doesn't understand the content of the zipped, binary-looking stuff, therefore it starts sieving again from the beginning, adding uncompressed relations at the end of the compressed file. And you only realize that after wasting 30 minutes of "special q" search. Suggestions? Can I decompress them by hand? (standard compression algorithm? I can look into the source, but I had to ask first). Do I have to look for a 64-bit yafu which uses compressed relations? Can I use any kind of command line switch as mentioned above? Can you (bb) implement one? (in case is not already and I am stupid, but I didn't find it in the docs). (that is the reason I posted in feature requests thread). Last fiddled with by LaurV on 2016-11-08 at 04:06 Reason: edited the log, you don't need all the story
 2016-11-08, 04:17 #217 bsquared     "Ben" Feb 2007 1100110011002 Posts The compression is a msieve thing, yafu doesn't have anything to do about it. Other than, of course, the fact that it uses msieve routines for post processing :) For the reasons you mention, none of the yafu builds that I've offered over the years use compression in the msieve portion, as far as I can remember. But it's been a real long time since I've built 32-bit yafu and I can't remember what I did the last time I included one in a download file. If it uses compression then it wasn't intentional... If you build your own yafu then you'll need to remember to build msieve with NO_ZLIB=1. As for the mess you currently have on your hands, I can't provide a recipe for unraveling it. Maybe you'll need to continue with factmsieve or something that uses a ZLIB enabled msieve.
 2016-11-08, 07:49 #218 LaurV Romulan Interpreter     Jun 2011 Thailand 5×11×157 Posts Thanks for your very fast reply. You already helped me with the msieve tip and the magic word "zlib"! I was skimming the yafu code last evening without finding the culprit and then gave up as I was tired from job. Now I know how to do.
 2016-11-08, 08:57 #219 LaurV Romulan Interpreter     Jun 2011 Thailand 5×11×157 Posts yarrr Thanks again B2. Code: nfs: commencing msieve linear algebra commencing linear algebra read 3759956 cycles cycles contain 12204676 unique relations read 12204676 relations using 20 quadratic characters above 536870840 building initial matrix memory use: 1579.0 MB read 3759956 cycles matrix is 3759778 x 3759956 (1133.2 MB) with weight 350932464 (93.33/col) sparse part has weight 255697659 (68.01/col) filtering completed in 2 passes matrix is 3757164 x 3757342 (1133.0 MB) with weight 350847953 (93.38/col) sparse part has weight 255679362 (68.05/col) matrix starts at (0, 0) matrix is 3757164 x 3757342 (1133.0 MB) with weight 350847953 (93.38/col) sparse part has weight 255679362 (68.05/col) saving the first 48 matrix rows for later matrix includes 64 packed rows matrix is 3757116 x 3757342 (1099.8 MB) with weight 281198460 (74.84/col) sparse part has weight 250723726 (66.73/col) using block size 65536 for processor cache size 3072 kB commencing Lanczos iteration (2 threads) memory use: 907.2 MB linear algebra at 0.0%, ETA 17h50m757342 dimensions (0.0%, ETA 17h50m) checkpointing every 210000 dimensions342 dimensions (0.1%, ETA 18h 0m) linear algebra completed 10385 of 3757342 dimensions (0.3%, ETA 18h 2m) (bitchy side of me: please note a printing anomaly at the one-before-last line, the overprinted line is shorted and it didn't erase to the end of line )
2016-11-08, 16:48   #220
chris2be8

Sep 2009

34·23 Posts

Quote:
 Originally Posted by LaurV (bitchy side of me: please note a printing anomaly at the one-before-last line, the overprinted line is shorted and it didn't erase to the end of line )
That's msieve again. I get output like that when running msieve directly (not called by YAFU).

Chris

 Similar Threads Thread Thread Starter Forum Replies Last Post ET_ Programming 0 2018-11-01 14:57 Dubslow YAFU 4 2012-03-31 03:07 Xyzzy Lounge 23 2011-03-08 17:50 ixfd64 Software 10 2010-05-31 15:21 rogue GMP-ECM 4 2009-11-23 15:07

All times are UTC. The time now is 01:22.

Wed Aug 12 01:22:49 UTC 2020 up 25 days, 21:09, 1 user, load averages: 1.60, 1.79, 1.80