Quick question--if I compile the latest revision of msieve (SVN 916), will that include the updated linear algebra code that significantly sped up that portion of the work?

On a related note, I compiled SVN 916 for my AMD64 with and without CUDA and ECM. I haven't fully tested them--only used the CUDA/ECM exe to perform stage 1 & 2 polynomial searching.

Edit: Just wanted to note that this was compiled on WinXP 32-bit with -march=native.

[QUOTE=WraithX;346263]
At this point, I'm not sure if the Windows builds can handle large jobs. I'm still trouble shooting my MinGW64 build. It turns out that it works just fine on a GNFS139 I had sitting on my hard drive. That only had a 3.02GB dat file. However, neither my MinGW64 nor Brian's Visual Studio build have worked on the L1911 24GB dat file. I think they read in all of the relations just fine, there just seem to be strange problems after that.[/QUOTE]

I compiled without large address awareness, which I imagine might cause problems with large data sets. I have sent Jeff a version compiled with large address awareness, which might be worth trying. I have also attached it here (hoping the mods won't mind).

The version on the website how now been updated (same filename but newer binary inside)

[QUOTE=Brian Gladman;346300]I compiled without large address awareness, which I imagine might cause problems with large data sets. I have sent Jeff a version compiled with large address awareness, which might be worth trying. I have also attached it here (hoping the mods won't mind).[/QUOTE]

At this point I've tried your Svn915 and your Svn915-Large-Address-Aware(LAA) and they have both crashed in the same spot on L1911, which is a GNFS167 with a 24GB dat file. Actually, I have also tried both of your binaries on a GNFS139 which had a 3GB dat file. All four tests crashed in the same spot, right after "Reading all ideals from disk". I dont' think LAA needs to be used in a 64-bit binary. That should only be needed with 32-bit binaries. One thing I'm wondering about, why do all of your compiles say i7? Could you try to compile a Core-2 binary and let me test that out? I don't know if there might be some instruction getting compiled in that my processors don't like, or if your binaries aren't working for me for some other reason. Speaking of which, can anyone else test this out to see if it will get to the linear algebra phase, or if it crashes before then?

Here is my log file from your SVN915 (both regular and LAA produced the same log file) on the GNFS139, in case this can help with any troubleshooting steps we might need to take. And, to hopefully make clear what I am thinking, the crashes I'm seeing from Brian's Visual Studio Win64 binaries are different from the crashes I'm seeing with my MinGW64 Win64 binaries.

[QUOTE=WraithX;346348]At this point I've tried your Svn915 and your Svn915-Large-Address-Aware(LAA) and they have both crashed in the same spot on L1911, which is a GNFS167 with a 24GB dat file. Actually, I have also tried both of your binaries on a GNFS139 which had a 3GB dat file. All four tests crashed in the same spot, right after "Reading all ideals from disk". I dont' think LAA needs to be used in a 64-bit binary. That should only be needed with 32-bit binaries. One thing I'm wondering about, why do all of your compiles say i7? Could you try to compile a Core-2 binary and let me test that out? I don't know if there might be some instruction getting compiled in that my processors don't like, or if your binaries aren't working for me for some other reason. Speaking of which, can anyone else test this out to see if it will get to the linear algebra phase, or if it crashes before then?

Here is my log file from your SVN915 (both regular and LAA produced the same log file) on the GNFS139, in case this can help with any troubleshooting steps we might need to take. And, to hopefully make clear what I am thinking, the crashes I'm seeing from Brian's Visual Studio Win64 binaries are different from the crashes I'm seeing with my MinGW64 Win64 binaries.[/QUOTE]

I agree that Large address aware should not matter for x64 but I wondered if there was some idiosyncracy that made it necessary. It would be useful to know the size of the smallest example that produces a crash and what the size of the data is for this example when compressed. I am not able to cope with a 24 GB download but I might be able to debug a much smaller example if one exists that will make it reasonable to download the data (~1 GB max).

Here is msieve compiled for Core2:

[QUOTE=Brian Gladman;346364]It would be useful to know the size of the smallest example that produces a crash and what the size of the data is for this example when compressed. I am not able to cope with a 24 GB download but I might be able to debug a much smaller example if one exists that will make it reasonable to download the data (~1 GB max).

Here is msieve compiled for Core2:[/QUOTE]

I've tried the Core2 compile and it too crashed on my C139 in the same spot as the earlier binaries. I then created a test GNFS number that is a C120 = P60*P60. I ran it through factmsieve.py with ggnfs and an old version of msieve. The gathering of relations and the msieve post-processing were all finished in about 5 hours. The dat file is 1.12GB and I was able to .tar.bz2 the .dat, the .ini, and the .fb file down to 465MB. When I ran this number through your SVN915 i7 and Core2 binaries I get a crash 2 minutes after starting the command "msieve -v -nc" in the same place as my previous crashes, ie right after "reading all ideals from disk". Brian, I can upload this test case to my web site, or I can give you access to my ftp server. PM me with how you would like to get this test case and I can PM you back the details of how/where to get it. Or, if you'd like a smaller test case I can probably make one of those too.

I can replicate the issue
 

[QUOTE=WraithX;346263]I downloaded the svn915 from Jeff's site a little while ago. Unfortunately, even this version of msieve crashed on me while working on L1911. This time it crashed on me in a slightly different place. I'm attaching the log to see if anyone can see what might have been the problem this time.
[/QUOTE]
I ran SVN 915 and I can confirm using a different number the one that is in the I seven test zip. It crashed in exactly the same place after about 5 minutes of running. I have attached my log

[QUOTE=Speedy51;346513]I ran SVN 915 and I can confirm using a different number the one that is in the I seven test zip. It crashed in exactly the same place after about 5 minutes of running. I have attached my log[/QUOTE]

What are the inputs for this example and where are they? Are they small enough to contemplate a download?

[QUOTE=WraithX;346422]I've tried the Core2 compile and it too crashed on my C139 in the same spot as the earlier binaries. I then created a test GNFS number that is a C120 = P60*P60. I ran it through factmsieve.py with ggnfs and an old version of msieve. The gathering of relations and the msieve post-processing were all finished in about 5 hours. The dat file is 1.12GB and I was able to .tar.bz2 the .dat, the .ini, and the .fb file down to 465MB. When I ran this number through your SVN915 i7 and Core2 binaries I get a crash 2 minutes after starting the command "msieve -v -nc" in the same place as my previous crashes, ie right after "reading all ideals from disk". Brian, I can upload this test case to my web site, or I can give you access to my ftp server. PM me with how you would like to get this test case and I can PM you back the details of how/where to get it. Or, if you'd like a smaller test case I can probably make one of those too.[/QUOTE]

Thanks to David's effort to provide me with a Windows test case that leads to a crash, I have found that the exception occurs at line 55 in filter.c because filter->lp_file_size is zero and no memory is allocated.

At line 159 of singleton.c the savefile msieve.dat.lp has been written and closed and the call "filter->lp_file_size = get_file_size(buf);" is used to get its size.

BUT get_file_size() doesn't find the file with this name (nor with ".gz" added) so zero length is returned. I don't yet know why as the file is present.

I have a compiled version of SVN916 for Core-i7 64bit with and without both CUDA and ECM if anybody is interested. I tested with both polynomial selection and relation sieving briefly and didn't get any crashes. If you want one of them, just PM and I'll be happy to email it to you.

Ben

[QUOTE=Brian Gladman;346578]Thanks to David's effort to provide me with a Windows test case that leads to a crash, I have found that the exception occurs at line 55 in filter.c because filter->lp_file_size is zero and no memory is allocated.

At line 159 of singleton.c the savefile msieve.dat.lp has been written and closed and the call "filter->lp_file_size = get_file_size(buf);" is used to get its size.

BUT get_file_size() doesn't find the file with this name (nor with ".gz" added) so zero length is returned. I don't yet know why as the file is present.[/QUOTE]

I have found the issue - I was compiling for a UNICODE character set so the ASCII file names were not being recognised. I will shortly pass a corrected version to Jeff.

Thanks to WraithX for his help in finding this. I have added the x64 i7 version below as a ZIP file.

Brian