[QUOTE=Batalov;165847]
OK. Will sieve some more; it's probably a marginal matrix. For the same size I usually sieved just a bit more. Tonight started with 46M raw relns, usually sieve to 50-52M. This is a 29-bit project.[/QUOTE]
Do you trust the stability of the machine? Has it completed smaller matrix jobs?

Yes, a few 27-bit ones, but of course one can never be too sure (it's been up for only a week). Now, this 29-bit run itself is a test for the upcoming 30-bit project. I am now filtering on the mature set of relns (50M, 46M unique and -nc1 10000000,10000000). The BL will only take 13 hours for this one, so in the evening I'll follow up on this.


EDIT: to be sure, I am doing it with msieve 1.39, the noble Rossinant. So far, so good.

Okay. Also, note that in v1.40 the two arguments to -nc1 are

- the filtering bound (both rational and algebraic)
- the number of relations to read in

This is a change from earlier versions, and was put in because more people seem to be heavily oversieving.

I will RMA this CPU. It produces the "submatrix is not invertible" with 4, 3 and even 2 threads. But it now went through ~8% of BL without glitches one 1 thread. (don't even ask: everything is set to standard.) And the box passed a night of memtest and a night and day of mprime at full load. Weird. So msieve could be the "new mprime" for sophisticated testers.

I probably can find which core is busted by taskset, but why bother. One must be busted.

I am sure that people have used the 6Mb cache and block size 65536 - I've seen it in logs. This cannot be the reason, or can it? Probably not. I've recomplied msieve-1.39 with forcing the 1Mb cache - no difference; still fails.

If it's hardware-related, it may not be the CPU; msieve LA pounds the bus wires very hard, and multiple threads may cause marginal memory accesses if all those memory channels are maxed out. Could you try multithreaded with some memory sticks pulled out? Is the temperature okay?

I'll try it tonight, right now I am not at that computer, just ssh.

I think I even found the bad core.
[FONT=Courier New]./msieve -s t.dat -i t.ini -l ggnfs.log -v -nf t.fb -t 3 -ncr[/FONT]
[FONT=Courier New]taskset -pc 0-2 <msieve_pid>[/FONT]
[FONT=Verdana]and it is stable (EDIT: no, not really). Use any other triple and the "submatrix is not invertible" very soon. [/FONT]
The temps are (via sensors):
[code]it8718-isa-0228
Adapter: ISA adapter
in0: +1.34 V (min = +0.00 V, max = +4.08 V) <-- this is CPU v
in1: +2.10 V (min = +0.00 V, max = +4.08 V) <-- this is memory
in2: +3.36 V (min = +0.00 V, max = +4.08 V)
in3: +2.94 V (min = +0.00 V, max = +4.08 V)
in4: +3.09 V (min = +0.00 V, max = +4.08 V)
in5: +3.23 V (min = +0.00 V, max = +4.08 V)
in6: +4.08 V (min = +0.00 V, max = +4.08 V)
in7: +2.67 V (min = +0.00 V, max = +4.08 V)
in8: +3.18 V
fan1: 1956 RPM (min = 0 RPM)
fan2: 0 RPM (min = 0 RPM)
fan3: 0 RPM (min = 0 RPM)
fan5: 0 RPM (min = 0 RPM)
temp1: +36.0C (low = +127.0C, high = +127.0C) sensor = transistor
temp2: +37.0C (low = +127.0C, high = +127.0C) sensor = thermal diode
temp3: +52.0C (low = +127.0C, high = +127.0C) sensor = transistor
cpu0_vid: +1.550 V <-- [I]this was said to be wrong in other threads. What is it?[/I][/code]
What sucks is that k10 support is [URL="http://lm-sensors.org/wiki/Devices"]yet to be written[/URL]. Damn.

From '[I]True lies'[/I]:
[I]Helen[/I]: Have you ever killed anyone?
[I]Harry[/I]: Yeah, but they were all bad.

I've made some changes to BIOS, but they were all good. I think.
1. Yeah, I removed two sticks for now. It's 4Gb for the time being.
2. Reset CMOS. Then set everything again. 15x multiplier, Sideport video memory (don't wont to steal from the main memory), Enable USB keyboard and mouse.
3. Instead of screwing around with DDR2 settings, I've enabled "EPP mode" and "Set by EPP". Now the memory voltage still set itself to 2.1v but I think timings were set to 5-5-5-15 and then something a bit different in what follows. (Last time I've set all these to 5-5-5-15 but then some other stuff from SPP which may have been wrong for 1066. SPP usually deals with 800 compatibility.)
4. Set the CPU fan to max (Disable smart fan.)
5. Made a mental note to buy a mini-cooler for memory. It was warm.

The max temp (the one that was 55C which doesn't look bad for Intel but here - it may have been) dropped to 46C. The CPU fan is at 3000rpm all the time.

ran memtest for an hour. It was fine before and is still fine.

booted openSUSE. Ram mprime for an hour in torture #1 (small size FFT, torture for the CPU). OK.

Continuing BL in 4 threads. So far, so good. Maybe, I won't RMA after all (or maybe those 2 sticks that I removed.) Feeling better.

[QUOTE=jasonp;165908]Okay. Also, note that in v1.40 the two arguments to -nc1 are

- the filtering bound (both rational and algebraic)
- the number of relations to read in

This is a change from earlier versions, and was put in because more people seem to be heavily oversieving.[/QUOTE]

I know that Jason knows; and entirely agree that it is remarkable how
well msieve does. I spent some three years fiddling with running snfs jobs
using Steffi's version of the CWI filter, with lots of pointers on "best practice"
from Peter. Filtering is complicated, especially if/when one wants to keep
matrices from getting too hard, using oversieving.

Still. There's not much difference between not reading in some of the
relations found, as a solution (temporary measure, for sure) for oversieving;
and just not sieving so much. We have msieve pre-processing with duplicate
removal. Seems to me that if/when msieve reports that oversieving has
produced too many relations, we should stop and run a separate stand-alone
program (like duplicate removal?) to trim the heaviest relations. The weight
should match the bounds used in filtering (primes above 720K, for the last
step), if possible; but if that's too intensive, something arbitrary and large
(for CWI format, we used primes above 10M). That would be a better use
of (for example) the CPU cycles needed to run an extra 200M range of q's,
than just throwing away 70% of the extra relns computed.

Any chance someone with the needed skills could check-out whether this
would be a plausible project? We're not looking at c90's; the code ought
to be able to deal with (say) 500,000,000 nonduplicate relations (for
double 32-bit large primes), so large files, and not too much memory use.
Or maybe one could make two runs, each with half of the relations, so
250M nonduplicate. Feed in 250M relations, write a file with the 225M
lightest relations, on primes above 5M-or-so. -Bruce

I was going to add more sophisticated excess removal for v1.40 but my time was limited and bug fixes have piled up in the meantime. I'd prefer that what you propose is folded right into the library, so that all the jobs undertaken benefit from it being available without the need to add even more utilities. UNless someone really wants to move forward on this, I'll put it on the short list for v1.41

Seems like something I could try, primarily because I've invested in the same. But anyone else so inclined, please go forward as well - because it will take me indefinite time (very busy lately).

[U]Re: the 'matrix non invertible' story[/U]. Bottom line: no bugs in msieve, on the contrary it served very well to find stress points in my new hardware. That BL and sqrt are now finished. Because it is such a good stress test, I will now redo my earlier snfs-225 on the same Phenom (it will serve as a 3-4-day long stress test; will restore full memory, too, and test it all).

--Serge

[QUOTE=jasonp;165997]I was going to add more sophisticated excess removal for v1.40 but my time was limited and bug fixes have piled up in the meantime. I'd prefer that what you propose is folded right into the library, so that all the jobs undertaken benefit from it being available without the need to add even more utilities. UNless someone really wants to move forward on this, I'll put it on the short list for v1.41[/QUOTE]

Yes, the "Optimizations to NFS with huge excess will have to wait until v1.41"
is over on the announcement thread; I had thought that I was describing
a non-sophisticated way for trimming some excess. What I missed was from
this thread, that 1.41 is scheduled for an early release.

Another non-sophisticated issue I was wondering about, which also involves
a large disk write that I had thought was out-side of the spirit of msieve,
is isolating singleton removal. If I'm recalling correctly, for these large
data sets, msieve is happier with a fresh start on a file with duplicates
removed. But relations that either have a prime that doesn't occur in
any other relation (a singleton), or that become singletons after previous
rounds of singleton removal, is similarly making references to relations
in subsequent filtering more complicated than needed. Back on RSA130,
before there was filtering, we referred to these early steps as "weeding";
subsequently post-RSA512, referred to as "repeated singleton removal".

Back on the heaviest-weight removal, which may be one of the options
included along with sophisticated removal (like higher merges, above
merge 2 ... there was a limit of 10 for a while, removed in later filters,
up to the limit of the algorithm ... 22 or so?). We have methods that make
choices, many of which are (very!) hard to compare intelligently. But
duplicate removal and singleton removal doesn't involve choices, those
relations are not useful (duplications, and relns that become singletons).
For example, would 1.39 filtering run better if both duplicates and singletons
were removed as a pre-computation?

Singletons are already removed in msieve (of course!), it's just a question
of writing a file, clearing pointers, and starting over with a nontrivially
smaller data set. -Bruce