[QUOTE=jasonp;133040]The msieve source is available along with a windows binary, and is where the common.h comes from. The library source ships with a VC9 build project maintained by Brian Gladman. It's as multi-platform as I could make it.

I didn't release the source for the conversion utility on my web page, since it was written as a special request and requires the msieve library source and a binary .lib or .a library file (will post the source tonight). GMP is not required to build the new converter, although Greg's version is much older, and requires the GGNFS and GMP source.

The input format for the new converter is as follows:

- worktodo.ini should just contain one line with the number

- the factor base file should just contain

N <number to be factored>
R0 <low-order rational polynomial coefficient>
R1 <high-order coefficient>
A0 <low-order algebraic coefficient>
..
A6 <high-order algebraic coefficient>
FRMAX <rational factor base bound>
FAMAX <algebraic factor base bound>

It can contain a lot more, but this is the minimum needed to make the conversion binary work. Note that if memory serves you negate all of the relation 'b' values to deal with the CWI suite's conventions, so you may have to negate some poly coefficients to make it work (GGNFS and msieve use the 'a-br' convention for relations, whereas the CWI suite uses the 'a+br' convention).[/QUOTE]


Thanks. Please note that #include <common.h> conflicts with
gcc. gcc has its own common.h file:

// Copyright (C) 1999-2002 Open Source Telecom Corporation.
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
//
// As a special exception to the GNU General Public License, permission is
// granted for additional uses of the text contained in its release
// of Common C++.
//
// The exception is that, if you link the Common C++ library with other
// files to produce an executable, this does not by itself cause the
// resulting executable to be covered by the GNU General Public License.
// Your use of that executable is in no way restricted on account of
// linking the Common C++ library code into it.
//
// This exception does not however invalidate any other reasons why
// the executable file might be covered by the GNU General Public License.
//
// This exception applies only to the code released under the
// name Common C++. If you copy code from other releases into a copy of
// Common C++, as the General Public License permits, the exception does
// not apply to the code that you add in this way. To avoid misleading
// anyone as to the status of such modified files, you must delete
// this exception notice from them.
//
// If you write modifications of your own for Common C++, it is your choice
// whether to permit this exception to apply to your modifications.
// If you do not wish that, delete this exception notice.

#ifndef CCXX_COMMON_H_
#define CCXX_COMMON_H_

#include <cc++/config.h>
#ifdef CCXX_EXPORT_LIBRARY
#include <cc++/export.h>
#endif
#include <cc++/missing.h>
#include <cc++/strchar.h>
#include <cc++/exception.h>
#include <cc++/thread.h>
#include <cc++/process.h>
#include <cc++/slog.h>
#include <cc++/file.h>
#include <cc++/socket.h>
#include <cc++/network.h>
#include <cc++/misc.h>
#include <cc++/digest.h>
#include <cc++/numbers.h>
#include <cc++/persist.h>
#include <cc++/serial.h>
#include <cc++/string.h>
#include <cc++/cmdoptns.h>
#include <cc++/url.h>
#include <cc++/xml.h>
#endif

[QUOTE=bdodson;133038]
sorry; inappropriate sarcasm. A few weeks ago when I adopted msieve
I noticed the need for the worktodo.ini file. I'm still not exacly clear why
it ought to be needed, but as someone that 10-15 years ago once ran
a mersenne range using prime95, it didn't/doesn't throw me off[/QUOTE]
The msieve library is at the same time a complete GNFS implementation (parts of which are sucky) and a recursive framework for batch factoring of one or more numbers. The entire batch can go into worktodo.ini or some other file, in which case each number is handled in turn. NFS unfortunately requires a lot more information which cannot be quickly generated from scratch, so that has to be cached in additional files. When doing the NFS postprocessing on a single number then you're in the awkward situation of having to give the library the same number in two places.

Bob, I've never seen gcc complaints about a header name conflict, or why C code would be pulling in C++ include files. Could your environment have different default compile flags compared to mine?

cwi2gg.c
 

The source to the cwi2gg binary is now available [url="www.boo.net/~jasonp/cwi2gg.c"]here[/url]. It requires a prebuilt msieve library and some of the library header files, but nothing else. Let me know if there are problems building it...

[QUOTE=bdodson;132769] ...[Jason's density inquiry] ...

3,536+ "sparse part has weight 874437208 (61.45/col)"

10,257- "sparse part has weight 858020543 (61.56/col)" and

10,257+ "sparse part has weight 952964006 (61.56/col)".

Current status is 57.4%, 29.6% and 6.5%. Maybe a month
and a half for 10,257+, ...

Is there a comparison with these and the known cases with
false dependencies?[/QUOTE]

OK, the 4th matrix is in,

3,547- "sparse part has weight 993915277 (61.86/col)"

a bit denser, certainly in the opposite direction from Tom's
caution on density nearer 57. Also larger,

16.067M^2

but this time the filtering shows (relatively) comfortable over-sieving
(unlike 10,257+). At C242 difficulty 260.99 this one is supposed to
be harder than the 257's (at 257).

Some bad news on timings, the c252 is at 70%, but at 27 of the
estimated 28 days. Adding a factor of (10/7) gives a new estimate
of 40 days, which sounds like May 24. That leaves the c241 257-
somewhere below 40 days, the C208 257+ at something like (1.9/1.5)*40,
with the new c242 at 16.067M^2 likely somewhere out past the
15.48M^2 for the 257+.

Other stat is "size score = 1.693893e-12, Murphy alpha = 2.209243,
combined = 9.010605e-13" for the poly rating. Compared with the
1.395729e-12 combined for the previous hardcase, 257+, there ought to
have been 1.39e-12/9.01e-13 = c. 1.54 more relns for 257+ than for
3,547- (if I'm reading these decimals correctly). Instead, I got way
more, which perhaps confirms the over-sieving. -Bruce

PS - all four using 4-threads.

Here's the announcement Bruce alluded to over [URL="http://www.mersenneforum.org/showpost.php?p=133264&postcount=2"]here[/URL]. The factorization of 12,241- has been completed.
[CODE]
12,241- (C260) = P66 * P194
p66:
146879693270660089750065267283010036196682888164682484908678010533
p194:
74873107005826461027456840546732017985015839954333825384171460000132499197947734733564329519019603384125346864931519118660986264214961894702863522511201474023708838835354065426450679143708619097
[/CODE]

Bruce completed the ECM pretesting, I did the sieving, and Richard completed the post-processing. A 18.7M^2 matrix was generated and solved successfully. Richard reports that each square root step took about 8 hours, and the factors were found on the third dependency.

Here's the log of the linear algebra & square root steps:
[CODE]Sat Mar 22 01:08:41 2008 commencing linear algebra
Sat Mar 22 01:42:32 2008 building initial matrix
Sat Mar 22 02:26:17 2008 memory use: 6947.3 MB


Sat Mar 22 03:40:43 2008 matrix is 18720804 x 18721052 (5170.3 MB)
with weight 1285924286 (68.69/col)
Sat Mar 22 03:40:43 2008 sparse part has weight 1168145840 (62.40/col)
Sat Mar 22 03:40:43 2008 matrix includes 64 packed rows
Sat Mar 22 03:40:43 2008 using block size 65536 for processor cache
size 4096 kB
Sat Mar 22 03:52:30 2008 commencing Lanczos iteration (4 threads)
Sat Mar 22 03:52:31 2008 memory use: 5547.0 MB

Sat Apr 26 11:50:51 2008 restarting at iteration 207228 (dim =
13104149)
Sun May 11 09:23:48 2008 lanczos halted after 296051 iterations (dim
= 18720802)
Sun May 11 09:25:13 2008 recovered 49 nontrivial dependencies
Sun May 11 09:25:20 2008 elapsed time 357:51:54

Sun May 11 09:27:07 2008 commencing square root phase

Mon May 12 01:06:39 2008 reading relations for dependency 3
Mon May 12 01:07:15 2008 read 9362213 cycles
Mon May 12 01:08:54 2008 cycles contain 32407242 unique relations
Mon May 12 01:30:10 2008 read 32407242 relations
Mon May 12 01:41:20 2008 multiplying 42565700 relations
Mon May 12 04:13:58 2008 multiply complete, coefficients have about
1247.77 million bits
Mon May 12 04:14:25 2008 initial square root is modulo 395611
Mon May 12 09:01:24 2008 prp66 factor:
146879693270660089750065267283010036196682888164682484908678010533
Mon May 12 09:01:24 2008 prp194 factor:
74873107005826461027456840546732017985015839954333825384171460000132499197947734733564329519019603384125346864931519118660986264214961894702863522511201474023708838835354065426450679143708619097
Mon May 12 09:01:24 2008 elapsed time 23:34:20
[/CODE]

The log of the filtering step is in my archives. I'll retrieve and post it later today...

Greg

[QUOTE=frmky;133272]Here's the announcement Bruce alluded to over [URL="http://www.mersenneforum.org/showpost.php?p=133264&postcount=2"]here[/URL]. The factorization of 12,241- has been completed.
[/QUOTE]Nice!

:bow:

Paul

[QUOTE=frmky;133272]
The log of the filtering step is in my archives. I'll retrieve and post it later today...
[/QUOTE]

Here's the log of the filtering for 12,241- that I did here up to the point that it ran out of memory. At this point, Richard volunteered to complete it.

[CODE]Mon Mar 17 18:12:06 2008 Msieve v. 1.33
Mon Mar 17 18:12:06 2008 random seeds: cb7c74ac 9de22da8
Mon Mar 17 18:12:06 2008 factoring 10997338991237101669052555789135086631048469132167079175601564990736546875635453637787116099924623852328671942230474398812245482555847896811945602723714843777323127914355152190001139337968698673482733772849274629055901187581713787723561860855660952506650948701 (260 digits)
Mon Mar 17 18:12:09 2008 no P-1/P+1/ECM available, skipping
Mon Mar 17 18:12:09 2008 commencing number field sieve (260-digit input)
Mon Mar 17 18:12:09 2008 R0: -14697715679690864505827555550150426126974976
Mon Mar 17 18:12:09 2008 R1: 1
Mon Mar 17 18:12:09 2008 A0: -1
Mon Mar 17 18:12:09 2008 A1: 0
Mon Mar 17 18:12:09 2008 A2: 0
Mon Mar 17 18:12:09 2008 A3: 0
Mon Mar 17 18:12:09 2008 A4: 0
Mon Mar 17 18:12:09 2008 A5: 0
Mon Mar 17 18:12:09 2008 A6: 12
Mon Mar 17 18:12:09 2008 size score = 1.585606e-12, Murphy alpha = 1.332237, combined = 1.083642e-12
Mon Mar 17 18:24:50 2008 restarting with 153096911 relations
Mon Mar 17 18:24:55 2008 added 45828 free relations
Mon Mar 17 18:24:55 2008
Mon Mar 17 18:24:55 2008 commencing relation filtering
Mon Mar 17 18:24:55 2008 commencing duplicate removal, pass 1
... errors snipped ...
Mon Mar 17 18:52:01 2008 found 30908445 hash collisions in 153141746 relations
Mon Mar 17 18:52:01 2008 commencing duplicate removal, pass 2
Mon Mar 17 19:04:50 2008 found 108 duplicates and 153141638 unique relations
Mon Mar 17 19:04:50 2008 memory use: 973.5 MB
Mon Mar 17 19:05:48 2008 ignoring smallest 5487658 rational and 5485186 algebraic ideals
Mon Mar 17 19:05:48 2008 filtering rational ideals above 94961664
Mon Mar 17 19:05:48 2008 filtering algebraic ideals above 94961664
Mon Mar 17 19:05:48 2008 need 16459266 more relations than ideals
Mon Mar 17 19:05:48 2008 commencing singleton removal, pass 1
Mon Mar 17 19:33:56 2008 relations with 0 large ideals: 1084395
Mon Mar 17 19:33:56 2008 relations with 1 large ideals: 8254914
Mon Mar 17 19:33:56 2008 relations with 2 large ideals: 28009207
Mon Mar 17 19:33:56 2008 relations with 3 large ideals: 50033828
Mon Mar 17 19:33:56 2008 relations with 4 large ideals: 47051982
Mon Mar 17 19:33:56 2008 relations with 5 large ideals: 18707312
Mon Mar 17 19:33:56 2008 relations with 6 large ideals: 0
Mon Mar 17 19:33:56 2008 relations with 7+ large ideals: 0
Mon Mar 17 19:33:56 2008 153141638 relations and about 91579140 large ideals
Mon Mar 17 19:33:56 2008 commencing singleton removal, pass 2
Mon Mar 17 20:02:24 2008 found 18506999 singletons
Mon Mar 17 20:02:24 2008 current dataset: 134634639 relations and about 72156819 large ideals
Mon Mar 17 20:02:24 2008 commencing singleton removal, pass 3
Mon Mar 17 20:27:33 2008 relations with 0 large ideals: 1084395
Mon Mar 17 20:27:33 2008 relations with 1 large ideals: 7915991
Mon Mar 17 20:27:33 2008 relations with 2 large ideals: 25779279
Mon Mar 17 20:27:33 2008 relations with 3 large ideals: 44277071
Mon Mar 17 20:27:33 2008 relations with 4 large ideals: 40135240
Mon Mar 17 20:27:33 2008 relations with 5 large ideals: 15442663
Mon Mar 17 20:27:33 2008 relations with 6 large ideals: 0
Mon Mar 17 20:27:33 2008 relations with 7+ large ideals: 0
Mon Mar 17 20:27:33 2008 134634639 relations and about 103032699 large ideals
Mon Mar 17 20:27:33 2008 commencing singleton removal, pass 4
Mon Mar 17 20:53:49 2008 found 26307678 singletons
Mon Mar 17 20:53:49 2008 current dataset: 108326961 relations and about 74498773 large ideals
Mon Mar 17 20:53:49 2008 commencing singleton removal, pass 5
Mon Mar 17 21:14:46 2008 relations with 0 large ideals: 1084395
Mon Mar 17 21:14:46 2008 relations with 1 large ideals: 7359120
Mon Mar 17 21:14:46 2008 relations with 2 large ideals: 22336110
Mon Mar 17 21:14:46 2008 relations with 3 large ideals: 35882152
Mon Mar 17 21:14:46 2008 relations with 4 large ideals: 30547144
Mon Mar 17 21:14:46 2008 relations with 5 large ideals: 11118040
Mon Mar 17 21:14:46 2008 relations with 6 large ideals: 0
Mon Mar 17 21:14:46 2008 relations with 7+ large ideals: 0
Mon Mar 17 21:14:46 2008 108326961 relations and about 90134063 large ideals
Mon Mar 17 21:14:46 2008 commencing singleton removal, pass 6
Mon Mar 17 21:37:41 2008 found 19828612 singletons
Mon Mar 17 21:37:41 2008 current dataset: 88498349 relations and about 68757462 large ideals
Mon Mar 17 21:37:41 2008 commencing singleton removal, pass 7
Mon Mar 17 22:00:22 2008 found 5798183 singletons
Mon Mar 17 22:00:22 2008 current dataset: 82700166 relations and about 62802258 large ideals
Mon Mar 17 22:00:22 2008 commencing singleton removal, pass 8
Mon Mar 17 22:23:09 2008 found 1654614 singletons
Mon Mar 17 22:23:09 2008 current dataset: 81045552 relations and about 61133708 large ideals
Mon Mar 17 22:23:09 2008 commencing singleton removal, pass 9
Mon Mar 17 22:46:03 2008 found 466614 singletons
Mon Mar 17 22:46:03 2008 current dataset: 80578938 relations and about 60665960 large ideals
Mon Mar 17 22:46:03 2008 commencing singleton removal, final pass
Mon Mar 17 23:16:02 2008 memory use: 1533.9 MB
Mon Mar 17 23:16:02 2008 commencing in-memory singleton removal
Mon Mar 17 23:16:13 2008 begin with 80578938 relations and 68016037 unique ideals
Mon Mar 17 23:19:47 2008 reduce to 69710530 relations and 56908564 ideals in 19 passes
Mon Mar 17 23:19:47 2008 max relations containing the same ideal: 32
Mon Mar 17 23:20:05 2008 filtering rational ideals above 750000
Mon Mar 17 23:20:05 2008 filtering algebraic ideals above 750000
Mon Mar 17 23:20:05 2008 need 120350 more relations than ideals
Mon Mar 17 23:20:05 2008 commencing singleton removal, final pass
Tue Mar 18 00:20:12 2008 keeping 62383957 ideals with weight <= 20, new excess is 5496085
Tue Mar 18 00:22:20 2008 memory use: 1919.4 MB
Tue Mar 18 00:22:20 2008 commencing in-memory singleton removal
Tue Mar 18 00:22:33 2008 begin with 69710530 relations and 62383957 unique ideals
Tue Mar 18 00:25:08 2008 reduce to 69685513 relations and 62358938 ideals in 11 passes
Tue Mar 18 00:25:08 2008 max relations containing the same ideal: 20
Tue Mar 18 00:26:24 2008 removing 4715610 relations and 4315610 ideals in 400000 cliques
Tue Mar 18 00:26:28 2008 commencing in-memory singleton removal
Tue Mar 18 00:26:40 2008 begin with 64969903 relations and 62358938 unique ideals
Tue Mar 18 00:28:52 2008 reduce to 64755292 relations and 57826905 ideals in 10 passes
Tue Mar 18 00:28:52 2008 max relations containing the same ideal: 20
Tue Mar 18 00:29:56 2008 removing 3405744 relations and 3005744 ideals in 400000 cliques
Tue Mar 18 00:29:59 2008 commencing in-memory singleton removal
Tue Mar 18 00:30:10 2008 begin with 61349548 relations and 57826905 unique ideals
Tue Mar 18 00:32:00 2008 reduce to 61225585 relations and 54696419 ideals in 9 passes
Tue Mar 18 00:32:00 2008 max relations containing the same ideal: 20
Tue Mar 18 00:32:59 2008 removing 1318289 relations and 1164582 ideals in 153707 cliques
Tue Mar 18 00:33:01 2008 commencing in-memory singleton removal
Tue Mar 18 00:33:12 2008 begin with 59907296 relations and 54696419 unique ideals
Tue Mar 18 00:34:33 2008 reduce to 59887608 relations and 53512110 ideals in 7 passes
Tue Mar 18 00:34:33 2008 max relations containing the same ideal: 20
Tue Mar 18 00:34:55 2008 relations with 0 large ideals: 159911
Tue Mar 18 00:34:55 2008 relations with 1 large ideals: 1341246
Tue Mar 18 00:34:55 2008 relations with 2 large ideals: 6124530
Tue Mar 18 00:34:55 2008 relations with 3 large ideals: 14660687
Tue Mar 18 00:34:55 2008 relations with 4 large ideals: 19277850
Tue Mar 18 00:34:55 2008 relations with 5 large ideals: 13442993
Tue Mar 18 00:34:55 2008 relations with 6 large ideals: 4155712
Tue Mar 18 00:34:55 2008 relations with 7+ large ideals: 724679
Tue Mar 18 00:34:55 2008 commencing 2-way merge
Tue Mar 18 00:36:46 2008 reduce to 36343498 relation sets and 29968000 unique ideals
Tue Mar 18 00:36:46 2008 commencing full merge[/CODE]

Greg

[QUOTE=xilman;133273]Nice!

:bow:

Paul[/QUOTE]

The 792-bit SNFS I've just finished over at the elevensmooth forum required a matrix very close in size to the 165-digit 6^383+1 GNFS:
[code]
6+383 : Tue Mar 11 18:23:21 2008 matrix is 7731132 x 7731353 (2156.9 MB) with weight 572189969 (74.01/col)
2+1188: Thu May 1 09:13:45 2008 matrix is 7933391 x 7933638 (2168.0 MB) with weight 536309417 (67.60/col)
[/code]

Assuming that the ratio is the same, the matrix work (about 1200 hours on a quad-core, if I scale correctly from the bit of the log around the 'restarted' message) for that 864-bit SNFS would be enough for a 179-digit GNFS :wink: It is good to know that on an 8GB machine I'd run out of patience for the matrix job before I run out of space.

@frmky: would it be possible to run the GNFS benchmark from [url]http://www.mersenneforum.org/showpost.php?p=126056&postcount=1[/url] on various numbers of cores on the Barcelona? I am curious to see how much of an effect memory contention has on the Franke lattice siever, and have no Barcelona-based machine to hand.

Greg,

Could you change line 256 of common/util.c from

[code]
if (max_special >= 0x80000006) {
CPUID(0x80000006, a, b, c, d);
cache_size2 = 1024 * (c >> 16);
}[/code]to[code]
if (max_special >= 0x80000006) {
CPUID(0x80000006, a, b, c, d);
cache_size2 = 1024 * (c >> 16);
cache_size2 = MAX(cache_size2,
512 * 1024 * (d >> 18));
}[/code]and see if that correctly detects the L3 cache size on your newfangled Barcelona processors? The cache size detection has needed a surprising number of changes to keep up with the latest from Intel and AMD...

[QUOTE=jasonp;133350]
see if that correctly detects the L3 cache size on your newfangled Barcelona processors? [/QUOTE]

This returned "using block size 65536 for processor cache size 2048 kB" which is the correct size of the shared L3 cache.

BTW, to more easily investigate the effect of using different numbers of threads, I added code to output the total time required to complete the linear algebra. In case you (or others) want to use it, here's the patch. I think it's portable, but I'm not sure it'll work correctly in Visual C++...

[CODE]--- orig/msieve-1.35/common/lanczos/lanczos.c 2008-05-13 22:42:34.000000000 -0700
+++ msieve_135_orig/common/lanczos/lanczos.c 2008-05-07 15:39:14.000000000 -0700
@@ -12,6 +12,7 @@
--------------------------------------------------------------------*/

#include "lanczos.h"
+#include <time.h>

#define BIT(x) ((uint64)(1) << (x))

@@ -912,6 +913,9 @@
uint32 report_interval = 0;
uint32 next_report = 0;
uint32 next_dump = 0;
+ uint32 start_dim = 0;
+ time_t start_t;
+ double hrs_per_dim;

if (packed_matrix->num_threads > 1)
logprintf(obj, "commencing Lanczos iteration (%u threads)\n",
@@ -959,6 +963,7 @@
winv, n, &dim_solved, &iter, s, &dim1);
logprintf(obj, "restarting at iteration %u (dim = %u)\n",
iter, dim_solved);
+ start_dim = dim_solved;
}
else {
v0 = (uint64 *)xmalloc(n * sizeof(uint64));
@@ -988,7 +993,8 @@
}

/* perform the iteration */
-
+
+ start_t = clock();
while (1) {
iter++;

@@ -1170,9 +1176,13 @@
if (report_interval) {
if (dim_solved >= next_report) {
next_report = dim_solved + report_interval;
- fprintf(stderr, "linear algebra completed %u "
- "out of %u dimensions (%1.1f%%)\r",
- dim_solved, n, 100.0 * dim_solved / n);
+ hrs_per_dim = (double)(clock()-start_t)/(3600.0*CLOCKS_PER_SEC)/(dim_solved-start_dim);
+ fprintf(stderr, "LA completed %u "
+ "of %u dims (%1.1f%%), "
+ "%.2f of %.2f hrs left \r",
+ dim_solved, n, 100.0 * dim_solved / n,
+ hrs_per_dim*(n-dim_solved),
+ hrs_per_dim*n);
fflush(stderr);
}
}
[/CODE]

Thanks,
Greg