[quote=10metreh;204929]The 60-to-80-digit work was done from 1999 to 2003, and the QS software wasn't as good either.[/quote]
Plus, I think the DB used to run sequences automatically up to 70-80 digits or so when it had a free moment. Of course, "free moments" on the DB are completely unheard of, so that doesn't happen any more.

(I could be wrong, though--it just seemed like it was always automatically running some sort of sequence back then when I'd look at the worker status page.)

If that was the case, that would explain the sequences that didn't have enough ECM done on their last composite--those might have been done a little farther than whatever was imported from somebody's collection of Aliquot sequence files, and thus left without adequate ECM on their final iteration.

Most of my sequences don't have any as I often stop a sequence after about a minute of ecm and I did a number of sequences in the early days to over 80 digits but less than 100 (before we had the main data). Also when I entered the main c80 data from Wolfgang Creyaufmuller, I just added the elf files, so no ecm was done other than the standard 'quick ecm'.

In general it is hard to stop aliqueit right after ecm, so I think many sequences in general won't have any (unless it is specifically mentioned). Only those canceled during ggnfs or yafu would have full ecm.

[quote=10metreh;204190]When you're doing a subproject, you don't always want to stop exactly when the sequence reaches the target height, you want to wait until it reaches a large composite. I think maybe modifying the script to finish when a composite greater than 4/5 of the target size (in digits) is reached might be better.[/quote]
I've been thinking about this and might have come up with a solution:

What if we interject another script into the msieve (or yafu) call made by aliqueit, and have that script check the number that is being sent to msieve (or yafu)? If the number meets the 80% length, it tells the system to halt that instance of aliqueit - if it's smaller, it calls msieve (or yafu). I'm not sure how clean an exit we could get, but it might be a way to interrupt processing based on the composite cofactor length. All we would need to do with aliqueit is change the program call line to our interceptor script and have our interceptor call msieve (or yafu), if warranted.

I'm not going to be able to pursue this for a while, but wondered if it sounds like it is worth checking into. How easy and safe is it to halt aliqueit via a system call?

Take Care,
Ed

[quote=EdH;205882]What if we interject another script into the msieve (or yafu) call made by aliqueit, and have that script check the number that is being sent to msieve (or yafu)? If the number meets the 80% length, it tells the system to halt that instance of aliqueit - if it's smaller, it calls msieve (or yafu). I'm not sure how clean an exit we could get, but it might be a way to interrupt processing based on the composite cofactor length. All we would need to do with aliqueit is change the program call line to our interceptor script and have our interceptor call msieve (or yafu), if warranted.[/quote]
Sounds like a good idea. It'd also need to do GNFS, of course.
[quote=EdH;205882]I'm not going to be able to pursue this for a while, but wondered if it sounds like it is worth checking into. How easy and safe is it to halt aliqueit via a system call?[/quote]
I coded something up, (along with the necessary modifications to aliperl) but I haven't figured out a way to get it to make aliqueit exit. Still trying to see about that...

I have coded a version of aliqueit that stops after ecm is completed if the composite is above a given size. I could share this - problem is that I am not able to easily produce the working executables as with the full release of a new aliqueit - In fact I am having troubles in that my own executable has a reliance on cygwin which is not what I want.

Would this help? It is actually a pretty small change to the alqueit code.

[quote=Greebley;205908]I have coded a version of aliqueit that stops after ecm is completed if the composite is above a given size. I could share this - problem is that I am not able to easily produce the working executables as with the full release of a new aliqueit - In fact I am having troubles in that my own executable has a reliance on cygwin which is not what I want.

Would this help? It is actually a pretty small change to the alqueit code.[/quote]
Sure, why not. We'll see which works out better. :smile: Despite the cygwin requirement, it might be much better in the end.

I've got it to a working state...not perfect, but its definitely functional. (the modified aliperl.pl and the interceptors for the SIQS/GNFS workers are attached) It will need tweaking to work in Linux (definitely the command to kill everything...I think the rest should work, but no guarantees). aliqueit should be set to call the intercept programs instead of the actual programs, (the actual program names are set in their interceptors) and should have use_msieve_polyfind set to false (with that set to true, it tries to call the intercept script to do the poly find, which doesn't work right; factmsieve.py natively supports using msieve to do poly finding anyway, so just use that).
The biggest issue I know of is that with the way I can get Windows to kill all of the programs, it outputs an error message, which can be safely ignored. The way I got it to do exit in Windows doesn't send aliperl a sigint so that it knows to exit gracefully, preventing it from updating the DB, so I instead had aliperl give the interceptors the necessary info so that they can update the DB when necessary.

Thanks for running with it, Mini-Geek,

I had thought that Aliqueit stepped through the other applications prior to ggnfs, but remember now that there is a value that you can set to have ggnfs run. My original thought was for a single intercept script, but I can see now that three are needed for this format.

Although on the surface, it would seem that a variation in the Aliqueit program would be a better choice, I'm thinking what you have may be better, although harder to set up. If Aliqueit is altered, this alteration would be needed each time Aliqueit is revised. Also, Aliqueit will need direct communication with aliperl. The only way I can see it working well, is if it is implemented as an option on the command line for Aliqueit to abort after the ECM if the cofactor is > x. But, maybe that would be a manner for it to be implemented... something like -a 80 to tell it to abort. But then, how to tell aliperl that was why it aborted...

I haven't tested your new stuff on my linux systems here yet, but will soon, hopefully. I am going to add another option in the script, to remove the ggnfs directory, once it is no longer needed. I'll be adding this as something for my CD/USB project, and want the script to stay consistent. I now have a (n almost) working CD** and a fully working USB version. For the CD version, I need to have the ggnfs directories removed since everything is running in RAMDisk. And, any files/directories are lost at powerdown anyway.

I think we can probably iron out the kill issues (if any) between linux and Win and it seems we should be able to capture the error code prior to its display. When I get a chance I'll play a little.

**My CD version appears to only be having trouble with the screen saver right now, but I'm doing some further testing. It is possible that this is really a manifestation of running out of memory, which is why I need to get rid of the ggnfs folders after they are no longer needed. As it stands the CD version is isolated from all the other media, unless a USB drive is inserted while running, or the user elects to mount available hard drives. The system I'm testing it on right now only has 512Meg to allocate to everything and I'm trying to determine if that is what's giving me difficulty, which basically is no apparent activity and a blank screen. Unfortunately, it might just be busy running ggnfs and not really locked up or it may be out of RAM.

Take Care,
Ed

[quote=EdH;206009]**My CD version appears to only be having trouble with the screen saver right now, but I'm doing some further testing. It is possible that this is really a manifestation of running out of memory, which is why I need to get rid of the ggnfs folders after they are no longer needed. As it stands the CD version is isolated from all the other media, unless a USB drive is inserted while running, or the user elects to mount available hard drives. The system I'm testing it on right now only has 512Meg to allocate to everything and I'm trying to determine if that is what's giving me difficulty, which basically is no apparent activity and a blank screen. Unfortunately, it might just be busy running ggnfs and not really locked up or it may be out of RAM.[/quote]
I'd guess the latter, that it's out of RAM. That would be consistent with what I've seen reported on other Linux systems, and it's unlikely that GGNFS would lock it up that badly even if it's running at normal priority (since Linux seems to handle even that OK most of the time).

The tricky thing is, even deleting the GGNFS folders after they're done might not be enough. In my experience GGNFS relations files are big enough that any largish job would more than fill up the RAM. A C91 or C93 might fit in 512MB, but definitely not a C100. And that's assuming that none of the 512MB is being used by the LiveCD itself, which would not be the case in reality. The only solution to this that I can think of would be to set the GNFS cutoff really high on low-RAM systems (and try not to do sequences where they might need to do big jobs that would be really inefficient with QS :smile:).

[quote=mdettweiler;206012]The only solution to this that I can think of would be to set the GNFS cutoff really high on low-RAM systems (and try not to do sequences where they might need to do big jobs that would be really inefficient with QS :smile:).[/quote]
Even QS uses relations files. I'm not how big they are compared to GNFS files, but that's not necessarily a perfect fix.

[quote=Mini-Geek;206015]Even QS uses relations files. I'm not how big they are compared to GNFS files, but that's not necessarily a perfect fix.[/quote]
I'm pretty sure that QS relation files are a whole heck of a lot smaller than GNFS ones, though--probably enough so that even a C110 would still be no more than 100MB or so at the most.

But, agreed, it's not a perfect fix since efficiency is rather drastically cut. If you could mount a USB drive, or possibly a hard disk if there's one available, that would solve the problem.

Well, I moved the CD to a system with 1.5G and another problem manifested. This time, all of a sudden all the files locked to read only and aliqueit had a fit. In light of the ggnfs size issues, maybe I need to rethink the CD project and finish working the USB one. I need to move both to a new thread as well.

As an added item, I notice that the CD project did retrieve a swap partition on both machines, although I really didn't want it to even "touch" the host hard drive. I don't know if there would be a way to use a portion of that for the ggnfs files...

Take Care,
Ed