[code]memory use: 3906.5 MB
saving the first 48 matrix rows for later
matrix is 9794898 x 9795123 (2873.1 MB) with weight 741367806 (75.69/col)
sparse part has weight 655209426 (66.89/col)
matrix includes 64 packed rows
using block size 65536 for processor cache size 4096 kB
commencing Lanczos iteration (4 threads)
memory use: 3070.6 MB
linear algebra completed 62469 of 9795123 dimensions (0.6%, ETA 336h34m)[/code]

[QUOTE=Joshua2;203137][code]
linear algebra completed 62469 of 9795123 dimensions (0.6%, ETA 336h34m)[/code][/QUOTE]
2 weeks on the nose seems about right.

If you have a crash - most likely due to a power failure, because msieve linear algebra is very reliable - make sure you restart with the -ncr flag, and not -nc or -nc2, so you don't overwrite the checkpoint (*.chk) file.

This may have changed since the last time I checked, which was several releases ago, but I wouldn't want to find out the hard way that it hadn't.

[QUOTE=Joshua2;203137][code]memory use: 3906.5 MB
saving the first 48 matrix rows for later
matrix is 9794898 x 9795123 (2873.1 MB) with weight 741367806 (75.69/col)
sparse part has weight 655209426 (66.89/col)
matrix includes 64 packed rows
using block size 65536 for processor cache size 4096 kB
commencing Lanczos iteration (4 threads)
memory use: 3070.6 MB
linear algebra completed 62469 of 9795123 dimensions (0.6%, ETA 336h34m)[/code][/QUOTE]

So 3.9 GB for building the matrix, and 3 GB for the matrix step itself. Thus, anything above ~c165 will probably need more than 4 GB.

BTW: What was the total relations count (raw and unique)?

Well, my guess for a week of LA was within a factor of two of being right. :lol:

[code] don't know which numbers u mean so...
found 23571633 hash collisions in 104490602 relations
added 4 free relations
commencing duplicate removal, pass 2
found 24543879 duplicates and 79946727 unique relations
memory use: 660.8 MB
reading ideals above 83820544
commencing singleton removal, initial pass
memory use: 1506.0 MB
reading all ideals from disk
memory use: 1401.5 MB
commencing in-memory singleton removal
begin with 79946727 relations and 75124962 unique ideals
reduce to 38203445 relations and 27416635 ideals in 18 passes
max relations containing the same ideal: 26
reading ideals above 720000
commencing singleton removal, initial pass
memory use: 753.0 MB
reading all ideals from disk
memory use: 1430.9 MB
keeping 36943195 ideals with weight <= 200, target excess is 205761
commencing in-memory singleton removal
begin with 38203451 relations and 36943195 unique ideals
reduce to 37901371 relations and 36640758 ideals in 14 passes
max relations containing the same ideal: 200
removing 3780522 relations and 3380522 ideals in 400000 cliques
commencing in-memory singleton removal
begin with 34120849 relations and 36640758 unique ideals
reduce to 33866739 relations and 33002482 ideals in 11 passes
max relations containing the same ideal: 194
removing 2760613 relations and 2360613 ideals in 400000 cliques
commencing in-memory singleton removal
begin with 31106126 relations and 33002482 unique ideals
reduce to 30946183 relations and 30479952 ideals in 9 passes
max relations containing the same ideal: 186
removing 1525944 relations and 1298396 ideals in 227548 cliques
commencing in-memory singleton removal
begin with 29420239 relations and 30479952 unique ideals
reduce to 29367177 relations and 29128103 ideals in 7 passes
max relations containing the same ideal: 180
relations with 0 large ideals: 672
relations with 1 large ideals: 754
relations with 2 large ideals: 11635
relations with 3 large ideals: 125059
relations with 4 large ideals: 740444
relations with 5 large ideals: 2630834
relations with 6 large ideals: 5886789
relations with 7+ large ideals: 19970990
commencing 2-way merge
reduce to 18229991 relation sets and 17990917 unique ideals
commencing full merge
memory use: 2084.7 MB
found 9830576 cycles, need 9799117
weight of 9799117 cycles is about 685979439 (70.00/cycle)
distribution of cycle lengths:
1 relations: 1397966
2 relations: 1371671
3 relations: 1304319
4 relations: 1122293
5 relations: 934181
6 relations: 783598
7 relations: 645832
8 relations: 521104
9 relations: 419278
10+ relations: 1298875
heaviest cycle: 22 relations
commencing cycle optimization
start with 50378352 relations
pruned 1034751 relations
memory use: 1722.2 MB
distribution of cycle lengths:
1 relations: 1397966
2 relations: 1401124
3 relations: 1347708
4 relations: 1141268
5 relations: 947827
6 relations: 784477
7 relations: 642297
8 relations: 513240
9 relations: 410654
10+ relations: 1212556
heaviest cycle: 22 relations
RelProcTime: 4943

commencing linear algebra
read 9799117 cycles
cycles contain 29145139 unique relations
read 29145139 relations
using 20 quadratic characters above 1073741568
building initial matrix
memory use: 3906.5 MB
read 9799117 cycles
matrix is 9798940 x 9799117 (2985.7 MB) with weight 924467746 (94.34/col)
sparse part has weight 665081683 (67.87/col)
filtering completed in 2 passes
matrix is 9794946 x 9795123 (2985.4 MB) with weight 924333719 (94.37/col)
sparse part has weight 665055415 (67.90/col)
read 9795123 cycles
matrix is 9794946 x 9795123 (2985.4 MB) with weight 924333719 (94.37/col)
sparse part has weight 665055415 (67.90/col)
saving the first 48 matrix rows for later
matrix is 9794898 x 9795123 (2873.1 MB) with weight 741367806 (75.69/col)
sparse part has weight 655209426 (66.89/col)
matrix includes 64 packed rows
using block size 65536 for processor cache size 4096 kB
commencing Lanczos iteration (4 threads)
memory use: 3070.6 MB
linear algebra at 0.0%, ETA 336h 3m5123 dimensions (0.0%, ETA 336h 3m)
linear algebra completed 190000 of 9795123 dimensions (1.9%, ETA 310h42m) dropping fast [/code]

[quote]begin with 79946727 relations and 75124962 unique ideals
reduce to 38203445 relations and 27416635 ideals in 18 passes[/quote]
Out of curiosity, was such a large cut (more than half !) on the number of usable relations expected at that early stage ?

The large number of singleton removal passes means that you don't have very much excess beyond the minimum needed to construct the matrix; if you had sieved for slightly longer then many more relations would survive. Unfortunately, experience shows that the matrix must be very large before the time saved solving a smaller matrix exceeds the extra sieving effort.

linear algebra completed 3853266 of 9795123 dimensions (39.3%, ETA 187h 2m) a day ahead of schedule?

[QUOTE=jasonp;203689]The large number of singleton removal passes means that you don't have very much excess beyond the minimum needed to construct the matrix; if you had sieved for slightly longer then many more relations would survive. Unfortunately, experience shows that the matrix must be very large before the time saved solving a smaller matrix exceeds the extra sieving effort.[/QUOTE]

Does that mean it might fail?

I don't think anything has gone wrong; it's just that most projects for inputs this large have had a little more sieving. Your matrix should work out fine.

ok 92.2% ETA 24 hrs going home for weekend result sunday/monday