[QUOTE=Speedy51;345296]I'm not sure if this idea was directed at me? I could try downloading a test case tomorrow and run it again. However if it errors out Brian will still need to download the files to replicate the error and see what's going on. Has anybody else run this code successfully on windows?

Brian if you read this can you please let me know if you have run a test with the.DAT file from [URL="http://www.chiark.greenend.org.uk/%7Etwomack/i7test.zip"]http://www.chiark.greenend.org.uk/~twomack/i7test.zip[/URL] it is almost the size of a CD

Thanks[/QUOTE]

I'm afraid not. Its already far to big to be a useful start in debugging.

What I really need is to see if there is a problem with a small factorisation - one that runs from scratch in minutes starting from a total input of a few kbytes or less.

I don't want to spend time debugging a large factorisation unless we know that the problem doesn't exist for small factorisations.

[QUOTE=Brian Gladman;345431]I'm afraid not. Its already far to big to be a useful start in debugging.

What I really need is to see if there is a problem with a small factorisation - one that runs from scratch in minutes starting from a total input of a few kbytes or less.

I don't want to spend time debugging a large factorisation unless we know that the problem doesn't exist for small factorisations.[/QUOTE]

I have now managed to run a small example and I find that running msieve causes an assertion failure in gmp-ecm while looking for small factors:

Assertion failed: __ecm_eulerphi (params->P) == params->s_1 * params->s_2, file ..\..\pm1fs2.c, line 2902

All GMP-ECM tests pass so I am not sure yet whether this is a problem in GMP-ECM or in the call used to enter the GMP-ECM library.

Thanks Brian for taking the time to look into this issue.

I've run into a strange problem and was hoping someone here can help me. I wanted to help out NFS@Home, and make sure I can run msieve post-processing, by trying to use one of the latest msieve svn's (>= 900) on a GNFS167 (L1911) which has a 24GB dat file with > 220M relations. I tried Brian's/Jeff's Win64 svn902, but (as noted above) that crashes during ecm. I tried Jason's Win32 svn900, but that crashed saying it couldn't reallocate ~2.6GB.

I then downloaded the latest svn (914) and tried to compile it in my MingW64/Msys environment. I ran the command: "make all Win=1 NO_ZLIB=1". I ran into an errror where I had to update/correct my pthreads headers/library/dll. Once I fixed that, the compilation proceeded all the way to the end. It didn't create the msieve.exe file because my ld.exe linker said that it did not recognize the --large-address-aware option. So, I figured since I was compiling with MingW64 on a Win64 system to be run on another Win64 system, that I could probably remove that option. I removed that option to the linker and then the compilation completed successfully. I moved that msieve.exe with its corresponding new W64-pthreads dll over to the machine where I was going to process L1911. It ran for about an hour, and then it crashed after producing some normal looking output with some strange content. Here is the end of the log:
[CODE]
Thu Jul 11 07:57:54 2013 commencing linear algebra
Thu Jul 11 07:58:01 2013 read 53171660 cycles
Thu Jul 11 07:58:02 2013 cycles contain 2 unique relations
Thu Jul 11 07:58:16 2013 read 2 relations
Thu Jul 11 07:58:16 2013 using 20 quadratic characters above 974902278
Thu Jul 11 07:58:16 2013 building initial matrix
Thu Jul 11 08:00:16 2013 memory use: 1825.6 MB
Thu Jul 11 08:00:31 2013 read 53171660 cycles
Thu Jul 11 08:00:40 2013 matrix is 132 x 53171660 (6287.8 MB) with weight 1914179751 (36.00/col)
Thu Jul 11 08:00:40 2013 sparse part has weight 1010261536 (19.00/col)
Thu Jul 11 08:01:14 2013 filtering completed in 2 passes
Thu Jul 11 08:01:14 2013 matrix is 117 x 317 (0.0 MB) with weight 11412 (36.00/col)
Thu Jul 11 08:01:14 2013 sparse part has weight 6023 (19.00/col)
Thu Jul 11 08:01:15 2013 matrix starts at (0, 0)
Thu Jul 11 08:01:15 2013 matrix is 117 x 317 (0.0 MB) with weight 11412 (36.00/col)
Thu Jul 11 08:01:15 2013 sparse part has weight 6023 (19.00/col)
Thu Jul 11 08:01:15 2013 matrix includes 98 packed rows
Thu Jul 11 08:01:15 2013 commencing Lanczos iteration
Thu Jul 11 08:01:15 2013 memory use: 0.0 MB
[/CODE]
The command line I used was:
msieve -nc -t 16 -v

The machine I'm running this on has Windows 7 Pro 64-bit, dual Xeon E5-2687W, 16 cores with hyperthreading turned off, and 64GB ram.

I'm attaching the whole log file below. Can anyone see what I might have done wrong? Perhaps my home compiled binary is to blame? Maybe I need to specify some other options to msieve to get this to work?

Is there plenty of disc space available and write-enabled? Those look a bit like some of the error messages I get when the disc is full

[QUOTE=fivemack;346033]Is there plenty of disc space available and write-enabled? Those look a bit like some of the error messages I get when the disc is full[/QUOTE]

This is on the second hard drive in the computer. There is 822GB free. I have turned off UAC and I am logged in as an administrator. The msieve.exe has full permissions for admins and users. Both the .cyc and .mat files are pretty small (3KB and 30KB). But that is probably just a symptom of the bigger problem.

I think that it is weird in the log file where it never has counts > 0 for relations with 1+ ideals, ie:
[CODE]
Thu Jul 11 07:57:31 2013 relations with 0 large ideals: 106239392
Thu Jul 11 07:57:31 2013 relations with 1 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 2 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 3 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 4 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 5 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 6 large ideals: 0
Thu Jul 11 07:57:31 2013 relations with 7+ large ideals: 0
[/CODE]
Could this help pinpoint the problem?

Note that 16 threads will probably be much slower than, say, 4 or 8 threads.

Edit: now that I looked through your logfile, it appears that the first singleton removal pass during the filtering wound up destroying the dataset. That shouldn't be happening. I think the only difference in your case from everyone else is that you are using MinGW64, which is IIRC the first time anyone has tried building Msieve this way.

[QUOTE=jasonp;346047]Note that 16 threads will probably be much slower than, say, 4 or 8 threads.

Edit: now that I looked through your logfile, it appears that the first singleton removal pass during the filtering wound up destroying the dataset. That shouldn't be happening. I think the only difference in your case from everyone else is that you are using MinGW64, which is IIRC the first time anyone has tried building Msieve this way.[/QUOTE]

I deleted the cyc and mat files and I have just started another run with:
msieve -nc -v
I'm wondering if it will work correctly when run single threaded. I'll let you know in about 9 hours. Until then, I'm also wondering if my MingW64 tool chain might be too out of date to understand the --large-address-aware option. I'm going to try to update my tool chain and compile it again. Then I'll try the new binary to see if it will work with multiple threads. I'll let you know that in about 10 hours. I thought I had compiled msieve with MingW64 before, but it looks like I did that back in 2008, and it looks like that was a 32-bit binary, so it must have been with regular MingW. Hopefully I can get this to compile with MingW64 because then there would be no dependency on any Visual C++ redistributables. Oh well, I'll let you know when I know more.

The --large-address-aware option is only to get a 32-bit binary to have a 3GB virtual address space, it's useless if you want a full 64-bit binary. Filtering is not threaded, so that doesn't matter here.

If you're willing to use gdb, the core of the singleton removal is in a single function and I can try to help you figure out what's going on there. We can take it to email if you're game; if using MinGW64 is the problem, then it would be nice to have another working build option in windows. The other possibility is to try using one of Microsoft's free compilers to build the source without ECM; even if it still doesn't work, you have a nicer symbolic debugger that can help figure out what's going on.

Ecm crashes for me in the latest binary as well. I assume it tries to use an instruction that my core 2 doesn't support.

Do you mean the 32-bit binary from sourceforge? My development machine is a core 2 so that's surprising. Or do you mean one of the binaries Jeff hosts?