ECMNet Feature Requests
 

I haven't touched ECMNet in about four years, so it's probably time for me to do a few updates to it. It doesn't appear that much needs to be done as I haven't gleaned much from what I've read in this forum (or others). I have a few things that I intend to do for the next release including:
[list][*]Removing support for sbfactor. AFAIK, it is no longer used by anybody.[*]Using the socket code I've been using for PRPNet.[*]Using the mail code I've been using for PRPNet, which added SMTP authentication support.[*]Adding support for -m in the client.[/list]
I have a few other ideas for the next release, so I'm posting them here to see what users are interested in. These include:
[list][*]Multi-thread the server[*]Use MySQL instead of the candidates file.[*]Allow client to grab multiple numbers to work on between communications with the server.[/list]
I'm open to other ideas. Please post them here.

Before giving up on MacOS X and installing my favorite Linux distro on my work's Mac, I remember having to make minor tweaks to the code and/or building system for the compilation on MacOS X to compile the *nix version (instead of the Windows version).

[quote]Use MySQL instead of the candidates file.[/quote]
The candidates file is easier to set up and edit than a MySQL server + database. It was pretty painless to set up an ECMNet server on my own computer to raise one of the XYYXF integers to t50.
Maybe, at least, sqlite could be used instead of MySQL ? :smile:

[QUOTE=debrouxl;232020]Before giving up on MacOS X and installing my favorite Linux distro on my work's Mac, I remember having to make minor tweaks to the code and/or building system for the compilation on MacOS X to compile the *nix version (instead of the Windows version).

The candidates file is easier to set up and edit than a MySQL server + database. It was pretty painless to set up an ECMNet server on my own computer to raise one of the XYYXF integers to t50.
Maybe, at least, sqlite could be used instead of MySQL ? :smile:[/QUOTE]

I'll fix any compile issues on OS X before I release it.

I'm not familiar with sqlite. MySQL is really easy to get going and runs on multiple platforms. I (and many others) use it for PRPNet without any problems. Any database I use would have to support transactions, i.e. no auto-commit.

[QUOTE=rogue;232048]I'll fix any compile issues on OS X before I release it.

I'm not familiar with sqlite. MySQL is really easy to get going and runs on multiple platforms. I (and many others) use it for PRPNet without any problems. Any database I use would have to support transactions, i.e. no auto-commit.[/QUOTE]

I did some further reading on sqlite and although it probably could work, the problem is that the PRPNet server is multi-threaded and I lock records within transaction allowing multiple clients to update concurrently. From what I have read, sqlite doesn't allow for record level locking which would cause clients to wait unnecessarily for work. ECMNet is a little different in that regard as there tends to be fewer clients.

On the other hand, sqlite might be an option for the client, which is apparently what it is more intended for, a client-side database. I don't see a need for a client-side database at this time.

[QUOTE=rogue;231941][list][*]Allow client to grab multiple numbers to work on between communications with the server.[/list][/QUOTE]

This sounds nice. I'm interested in a feature to connect yoyo@home to different ecmnet servers. For this it is requied, that I can fetch e.g. 1000 curves for a number and send done curves and found factors days/weeks later back to the server.
E.g. I get a file with <server> <number> <requied curves>, than I create a number of Boinc Wus out of it. When parts of them are finished (days/weeks later) I send them back.
Of course I have to filter them if they fit to my requirements, because I support only a defined range of digit-length and B1.

yoyo

[QUOTE=yoyo;232056]This sounds nice. I'm interested in a feature to connect yoyo@home to different ecmnet servers. For this it is requied, that I can fetch e.g. 1000 curves for a number and send done curves and found factors days/weeks later back to the server.
E.g. I get a file with <server> <number> <required curves>, than I create a number of Boinc Wus out of it. When parts of them are finished (days/weeks later) I send them back.
Of course I have to filter them if they fit to my requirements, because I support only a defined range of digit-length and B1.[/QUOTE]

I don't know the architecture of yoyo@home. I would need to understand that better before I know what you are asking for. If desired, we can talk about this via e-mail and then I can put the results of said discussions into this thread.

[QUOTE=rogue;231941]I haven't touched ECMNet in about four years, so it's probably time for me to do a few updates to it. It doesn't appear that much needs to be done as I haven't gleaned much from what I've read in this forum (or others).
[snip]
I'm open to other ideas. Please post them here.[/QUOTE]

When I use the ecmnet client, I pretty much always want to run multiple copies of it, all sharing most of their config information. In particular, I would like them to all be able to use the same config file. As far as I can tell, ecmnet makes this impossible.

I assume other people like to run multiple clients. What do you all do? Have a different directory for each client, each with a slightly different config file?

[QUOTE=jyb;232225]When I use the ecmnet client, I pretty much always want to run multiple copies of it, all sharing most of their config information. In particular, I would like them to all be able to use the same config file. As far as I can tell, ecmnet makes this impossible.

I assume other people like to run multiple clients. What do you all do? Have a different directory for each client, each with a slightly different config file?[/QUOTE]

The problem is the the client does i/o to a set of work files that are named based upon the ini settings. With multiple copies of the client running from the same directory, they would overwrite one another. That is the problem that needs to be solved in order to allow what you want to happen. I could add a command line switch that would append a value to the work file name, but if you other ideas, I'm listening.

[QUOTE=rogue;232232]The problem is the the client does i/o to a set of work files that are named based upon the ini settings. With multiple copies of the client running from the same directory, they would overwrite one another. That is the problem that needs to be solved in order to allow what you want to happen. I could add a command line switch that would append a value to the work file name, but if you other ideas, I'm listening.[/QUOTE]

Yes, in fact the problems are these:
[list][*]The client always looks for "ecmclient.cfg" in the current directory, so all clients have to have the same current directory. That would be fine, but:[*]The log file names are hardcoded, so the clients's log and factors files will all stomp on each other.[*]The suffix for work file names is set in the config file, so all clients will have the same suffix, meaning that they will all stomp on each other.[*]The machine ID is set in the config file and can't be overridden, so different client processes can't have different IDs.[/list]
So it seems that the only way to run multiple clients is for each client to have its own directory containing a slightly different config file, and each client process has to run out of its directory. Needless to say, this is a giant pain.

So I made some changes in my local copy that allow multiple clients to share the same config file without any stomping. They can all therefore run from the same directory. The changes are:
[list][*]Use the machine ID to identify each client.[*]Allow the machine ID to be overridden on the command line (with "-ID").[*]Append the machine ID string to the log and factors file names (e.g. ecmclient_myID.log).[*]Recognize a special token in the config file's server descriptor for the work file suffix. If the suffix is specified there as "$ID" then use the machine ID as the suffix. (E.g. work_myID.in).[/list]
So now I just set the suffix in the ecmclient.cfg to $ID, and I can fire off any number of clients from the same directory with "ecmclient -ID <string>" and they will all happily coexist (as long as each one has a different <string>, of course). I've been running it this way for the last year or so.

If you want my diffs, let me know.

[QUOTE=jyb;232237]Yes, in fact the problems are these:
[list][*]The client always looks for "ecmclient.cfg" in the current directory, so all clients have to have the same current directory. That would be fine, but:[*]The log file names are hardcoded, so the clients's log and factors files will all stomp on each other.[*]The suffix for work file names is set in the config file, so all clients will have the same suffix, meaning that they will all stomp on each other.[*]The machine ID is set in the config file and can't be overridden, so different client processes can't have different IDs.[/list]
So it seems that the only way to run multiple clients is for each client to have its own directory containing a slightly different config file, and each client process has to run out of its directory. Needless to say, this is a giant pain.

So I made some changes in my local copy that allow multiple clients to share the same config file without any stomping. They can all therefore run from the same directory. The changes are:
[list][*]Use the machine ID to identify each client.[*]Allow the machine ID to be overridden on the command line (with "-ID").[*]Append the machine ID string to the log and factors file names (e.g. ecmclient_myID.log).[*]Recognize a special token in the config file's server descriptor for the work file suffix. If the suffix is specified there as "$ID" then use the machine ID as the suffix. (E.g. work_myID.in).[/list]
So now I just set the suffix in the ecmclient.cfg to $ID, and I can fire off any number of clients from the same directory with "ecmclient -ID <string>" and they will all happily coexist (as long as each one has a different <string>, of course). I've been running it this way for the last year or so.

If you want my diffs, let me know.[/QUOTE]

Why not? Just e-mail them to me.

[QUOTE=jyb;232237]
So it seems that the only way to run multiple clients is for each client to have its own directory containing a slightly different config file, and each client process has to run out of its directory. Needless to say, this is a giant pain.

So I made some changes in my local copy that allow multiple clients to share the same config file without any stomping. They can all therefore run from the same directory. The changes are:
[LIST][*]Use the machine ID to identify each client.[*]Allow the machine ID to be overridden on the command line (with "-ID").[*]Append the machine ID string to the log and factors file names (e.g. ecmclient_myID.log).[*]Recognize a special token in the config file's server descriptor for the work file suffix. If the suffix is specified there as "$ID" then use the machine ID as the suffix. (E.g. work_myID.in).[/LIST][/QUOTE]
I do it slightly differently, in that I use different directories but identically the same config files for each client.First create the directories (I call them cl1, cl2, ..., cln) and in one of them create a config file. Then [B]hard link[/B] to that config file to the same name in each of the other directories.

Second create a shell script which contains lines:
[code]#! /bin/sh
cd /path/to/cl1
/path/to/ecmclient &
cd /path/to/cl2
/path/to/ecmclient &
cd /path/to/cl3
/path/to/ecmclient &
...
cd /path/to/cln
/path/to/ecmclient &
[/code]Third, run the shell script. Because the config files are hard linked together, editing any one immediately ensures that all the others are changed at the same time.

If you wish, you could get clever and write the script as a loop which runs from 1 to $1, thereby starting a user-selected number of clients. but I couldn't be bothered. If you wanted to get really clever, you could write a script (probably easier in Perl than sh) which writes customized config files in each directory, creating the directory if required. Again, I couldn't be bothered but if there's interest I could write one easily enough.

Paul

[QUOTE=xilman;232313]I do it slightly differently, in that I use different directories but identically the same config files for each client.First create the directories (I call them cl1, cl2, ..., cln) and in one of them create a config file. Then [B]hard link[/B] to that config file to the same name in each of the other directories.

Second create a shell script which contains lines:
[code]#! /bin/sh
cd /path/to/cl1
/path/to/ecmclient &
cd /path/to/cl2
/path/to/ecmclient &
cd /path/to/cl3
/path/to/ecmclient &
...
cd /path/to/cln
/path/to/ecmclient &
[/code]Third, run the shell script. Because the config files are hard linked together, editing any one immediately ensures that all the others are changed at the same time.

If you wish, you could get clever and write the script as a loop which runs from 1 to $1, thereby starting a user-selected number of clients. but I couldn't be bothered. If you wanted to get really clever, you could write a script (probably easier in Perl than sh) which writes customized config files in each directory, creating the directory if required. Again, I couldn't be bothered but if there's interest I could write one easily enough.

Paul[/QUOTE]

That all sounds rather hideous. Having to have separate directories in the first place is just ridiculously annoying. With the changes I detailed above, I can run any number* of clients out of the same directory, and all their log and work files can be in one place and easily identified. Plus, each running client can have a different ID, which is impossible if you're hard-linking the config files such that they're all the same. So if the server notifies me that I found a factor, I don't have to search in order to know which client found it.

And yes, I use a script to fire off a bunch of clients. But it's very simple because it doesn't need to cd anywhere.

*When I say any number, I really mean it. My home directory is shared by many machines via NFS, so I may have upwards of 80 cores each running its own copy of the client, all from the same directory.

[QUOTE]I'm open to other ideas. Please post them here.[/QUOTE]A Debian package would be very useful, perhaps integrated into the existing [URL="http://packages.debian.org/lenny/gmp-ecm"]GMP-ECM[/URL] package.

:max:

[QUOTE=Xyzzy;232495]A Debian package would be very useful, perhaps integrated into the existing [URL="http://packages.debian.org/lenny/gmp-ecm"]GMP-ECM[/URL] package.

:max:[/QUOTE]

I have absolutely no clue how to do such a thing.

[QUOTE=jyb;232493]That all sounds rather hideous. Having to have separate directories in the first place is just ridiculously annoying.[/QUOTE]Horses for courses. It works well for me.

Paul

[QUOTE=xilman;232511][QUOTE=jyb;232493]That all sounds rather hideous. Having to have separate directories in the first place is just ridiculously annoying.[/QUOTE]Horses for courses. It works well for me.

Paul[/QUOTE]

Oh, I don't doubt it. And please excuse the unnecessarily critical tone of my last post; I didn't intend it as such. With the right script it certainly doesn't require much work to start/stop a bunch of clients.

But every time you want to add new client don't you have to make yet another directory and yet another hard link? It was this management that I was referring to as annoying. And don't all clients end up with the same ID? I grant that that's not particularly onerous, I just think that the changes I detailed above make the client much more usable, with no drawbacks that I can see.

I have decided to modify the client to take a -id argument. This will be used to override the clientid in the cfg file. This clientid will be appended to the file name to maintain uniqueness.

[QUOTE]I have absolutely no clue how to do such a thing.[/QUOTE][URL]http://wiki.debian.org/HowToPackageForDebian[/URL]

There are many links similar to that one.

Debian packages can be used on Debian, Ubuntu and several other Linux systems. Basically it is just a wrapper that configures things and installs them.

We like that we can take a new Debian box and, as root, run:

aptitude install gmp-ecm

And then it is installed. Our next command (after we relinquish root privileges) can run ECM.

Currently, sending the ecmclient program either SIGTERM, SIGINT or SIGQUIT, stops the current curve immediately, before it completes.
What about adding another signal, such as SIGUSR1, for a delayed shutdown (including communication with the server) after the current curve completes ? :smile:

EDIT: and, since you're removing support for sbfactor, what about affecting the "memorytouse" parameter to GMP-ECM (-maxmem <n>) ?

[QUOTE=debrouxl;235172]Currently, sending the ecmclient program either SIGTERM, SIGINT or SIGQUIT, stops the current curve immediately, before it completes.
What about adding another signal, such as SIGUSR1, for a delayed shutdown (including communication with the server) after the current curve completes ? :smile:

EDIT: and, since you're removing support for sbfactor, what about affecting the "memorytouse" parameter to GMP-ECM (-maxmem <n>) ?[/QUOTE]

I will be adding support for the GMP-ECM -k option.

As for SIGUSR1, I don't know. Note that the ECMNet client does not link with GMP-ECM, but rather runs it via the system() function. My understanding is that both GMP-ECM and the ECMNet client get the signal, thus the client cannot control how GMP-ECM responds to that signal.

I don't know if it is possible, but GMP-ECM is not convient to pause,
especially using B1 very large such as 26e7 or 85e7 and after running 2 hours and press Ctrl-C the result will be lost.

And I use ECMNet about four years before, it always give me thousands of curves each time, but I have to shut down my computer each day, so is it possible to let the ECMNet client run less than 3 hours for each work unit.

I notice yoyo@Home split each curve to about 4 steps something like
ecm -save temp1.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp1.out -save temp2.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp2.out -save temp3.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp3.out -save temp4.out 11e7 -maxmem 1024 <in.num >>result.out
Is it possible to use this technical in ECMNet?

[QUOTE=wreck;235190]I don't know if it is possible, but GMP-ECM is not convient to pause,
especially using B1 very large such as 26e7 or 85e7 and after running 2 hours and press Ctrl-C the result will be lost.

And I use ECMNet about four years before, it always give me thousands of curves each time, but I have to shut down my computer each day, so is it possible to let the ECMNet client run less than 3 hours for each work unit.

I notice yoyo@Home split each curve to about 4 steps something like
ecm -save temp1.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp1.out -save temp2.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp2.out -save temp3.out 11e7 -maxmem 1024 <in.num >>result.out
ecm -resume temp3.out -save temp4.out 11e7 -maxmem 1024 <in.num >>result.out
Is it possible to use this technical in ECMNet?[/QUOTE]

There is the "frequency" option in the client to control how long between communications to the server. Are you referring to processing of curves that take more than a few hours per curve, so that you can start today and continue tomorrow or something like that.

In yoyo@home I wrote a wrapper arround gmp-ecm. It runs ecm with -chkpnt. This option periodicaly saves the state in phase 1 of the run. Phase 1 is roughly 80% of the total runtime. So during this phase checkpoints are generated.
These checkpoints are used to in restart cases. So if you shoutdown Boinc or your computer or if something crashes and Boinc restarts, the wrapper checks if a checkpoint file exists. If yes, it runs ecm with -resume to resume from the checkpoint.
yoyo

[QUOTE=debrouxl;235172]Currently, sending the ecmclient program either SIGTERM, SIGINT or SIGQUIT, stops the current curve immediately, before it completes.
What about adding another signal, such as SIGUSR1, for a delayed shutdown (including communication with the server) after the current curve completes ? :smile:
[/QUOTE]

Actually, your description of the current behavior isn't accurate. In fact, the current behavior appears to be exactly what you're asking for. When the ecmclient program gets one of those signals, it sets a flag, and the next time a curve completes it shuts down in an orderly fashion.

You may be getting confused by what's happening if you're running ecmclient as a foreground job in some shell and you do a Ctrl-C. In such circumstances, the shell sends a SIGINT to the entire foreground process group, so GMP-ECM is getting one as well, causing it to die immediately (and ecmclient to then clean up immediately). But if you send a SIGINT (or SIGTERM or SIGQUIT) to just the ecmclient process (e.g. from some other shell), then you should see the behavior you want.

[QUOTE=rogue;235183]I will be adding support for the GMP-ECM -k option.
[/QUOTE]

??

You already have support for the GMP-ECM -k option (see "gmpecmkvalue" in the .cfg file). But it would be really nice to directly support -maxmem. I've done that for my own use, and it's really nice (Paul L's ECMNet server has lately been handing out P+1 assignments with a B1 of 260e6, and GMP-ECM would otherwise want about 2.5 GB of memory for these). Let me know if you want me to send you my diffs.

@Rogue What is stopping you doing something similar to prpnet?

[QUOTE=jyb;235248]??

You already have support for the GMP-ECM -k option (see "gmpecmkvalue" in the .cfg file). But it would be really nice to directly support -maxmem. I've done that for my own use, and it's really nice (Paul L's ECMNet server has lately been handing out P+1 assignments with a B1 of 260e6, and GMP-ECM would otherwise want about 2.5 GB of memory for these). Let me know if you want me to send you my diffs.[/QUOTE]

I'm sorry. I had that reversed. I added support for -maxmem, not -k, which was already there.

[QUOTE=henryzz;235251]@Rogue What is stopping you doing something similar to prpnet?[/QUOTE]

Specifically what?

There are some fundamental differences between how PRPNet and ECMNet work. I intend to have them share a lot more code with the next release, but I don't think that a database is necessary for ECMNet. Maybe someone disagrees, but I don't see ECMNet servers having as many dedicated clients.

[QUOTE=rogue;235253]Specifically what?

There are some fundamental differences between how PRPNet and ECMNet work. I intend to have them share a lot more code with the next release, but I don't think that a database is necessary for ECMNet. Maybe someone disagrees, but I don't see ECMNet servers having as many dedicated clients.[/QUOTE]
I meant with stopping and starting.
As far as I remember that was a pretty early change in prpnet. I fail to understand how starting llr or pfgw is different to starting ecm.

[QUOTE=henryzz;235254]I meant with stopping and starting.
As far as I remember that was a pretty early change in prpnet. I fail to understand how starting llr or pfgw is different to starting ecm.[/QUOTE]

GMP-ECM didn't always write a checkpoint. If it does now or has the capability to, then I can support it.

[quote=jyb]You may be getting confused by what's happening if you're running ecmclient as a foreground job in some shell and you do a Ctrl-C. In such circumstances, the shell sends a SIGINT to the entire foreground process group, so GMP-ECM is getting one as well, causing it to die immediately (and ecmclient to then clean up immediately). But if you send a SIGINT (or SIGTERM or SIGQUIT) to just the ecmclient process (e.g. from some other shell), then you should see the behavior you want.[/quote]
You're correct, thanks :wink:

[QUOTE=rogue;235253]Specifically what?

There are some fundamental differences between how PRPNet and ECMNet work. I intend to have them share a lot more code with the next release, but I don't think that a database is necessary for ECMNet. Maybe someone disagrees, but I don't see ECMNet servers having as many dedicated clients.[/QUOTE]One of my ideas a while back was to use a proper database behind an ECMNET server. Not so much to support more clients, though that would be a welcome by-product but more to make admin tasks easier.

I'd also make the server multi-threaded or multi-processing. At the moment a single rogue client (ambiguity intentional!) can hang the server, thereby mounting a denial of service to other clients.

Paul

[QUOTE=xilman;235329]One of my ideas a while back was to use a proper database behind an ECMNET server. Not so much to support more clients, though that would be a welcome by-product but more to make admin tasks easier.

I'd also make the server multi-threaded or multi-processing. At the moment a single rogue client (ambiguity intentional!) can hang the server, thereby mounting a denial of service to other clients.

Paul[/QUOTE]

With PRPNet, I added a database and multi-threading at the same time. Doing so with ECMNet should be fairly easy. I was hesitant to make such changes because I didn't see the value. Since you see value in it and are you are one of the bigger users (that I know of), I'll add that for the next release.

I'm working again on the changes to ECMNet. I have a question for those of you who set up servers.

Does anyone use master/slave configurations?

In the PRPNet world, I don't support it because clients can d/l as much work as the server allows. While the client has work it is reserved. This means that no other client will get a PRP test that is reserved by another client.

In the ECMNet world, things are different. It is common to have multiple clients doing ECM curves on the same number at the same time. This is controlled via the "hour breakpoints" in the ecmserver.cfg. The server also does not track reservations as the client will continue doing curves on the number until it reaches the max. This means that a client could have outstanding work for days, weeks, or longer.

I have considered some changes so that the client could get workunits and then report back to the server when the workunits have completed or when a certain amount of work has been done. If I do this, then I could set up a reservation system which could also be used to vary the work a little more with the clients. For example, I could have it send this:

Type=P+1 Curves=3 B1=100000 Number=Number1
Type=P-1 Curves=1 B1=100000 Number=Number1
Type=ECM Curves=100 B1=100000 Number=Number1
Type=P+1 Curves=3 B1=100000 Number=Number2
Type=P-1 Curves=1 B1=100000 Number=Number2
Type=ECM Curves=100 B1=100000 Number=Number2
Type=P+1 Curves=3 B1=300000 Number=Number2
Type=P-1 Curves=1 B1=300000 Number=Number2
Type=ECM Curves=100 B1=300000 Number=Number2

etc.

until max work per client is exceeded.

The number of curves is evaluated as MIN(curves left for that B1, max per client for that B1). The client would return completed work when either:
a) all work completed
b) max time for server exceeded (for example, client does curves for 10 hours, then returns what was completed, the max time is configurable)
c) max work exceeded (work is a computed value based upon decimal length of the number and B1, when a curve is completed, work is incremented, max work is a configurable threshold).

I'm not proposing to get rid of the "hour breakpoints", but I want to allow the end user to have more control over the work they do and the allow the server admin more control over reservations.

Thoughts?

[QUOTE=rogue;239671]

[snip]

For example, I could have it send this:
[/QUOTE]

What is the "it" in that sentence? The server? I.e. are you just proposing that rather than sending a single number with a certain number of curves you send a somewhat longer list of work units to perform?

[QUOTE=rogue;239671]

I'm not proposing to get rid of the "hour breakpoints", but I want to allow the end user to have more control over the work they do and the allow the server admin more control over reservations.

Thoughts?[/QUOTE]

I think I must be misinterpreting your proposal because I don't see how it accomplishes either of those aims. The client still performs exactly the work that the server sends it, right? That work is now a longer list of numbers/curves than before, but in what way does the end user have more control?

And how does this relate to reservations? Is the idea that once the server sends work to one client, it is reserved (and therefore not sent to anyone else) until either the work is sent back as either released or completed, or the client times out? It's not clear that that would be a desirable change to make (I could probably be convinced), but either way it seems orthogonal to the proposal you've made (unless, again, I'm just not correctly interpreting your proposal).

[QUOTE=rogue;239671]Does anyone use master/slave configurations?
[/QUOTE]

I use them.

I use them to segregate work units in various ways. Odd Perfect Number has a server that hands out only roadblocks and another that hands out only small tasks. Users can prioritize based on interest and computer power.

While I was living in London, I used a slave server for administration of the OPN servers that stayed in the USA. I could easily get an updated ecmserver.ini, and I also reported factors to the master by hand creating factor lines in the slave's ecmserver.ini.

William

[QUOTE=jyb;240789]What is the "it" in that sentence? The server? I.e. are you just proposing that rather than sending a single number with a certain number of curves you send a somewhat longer list of work units to perform?[/QUOTE]

Yes. If the server has lots of small numbers to run curves on, then the client could attempt to factor more than one before communicating with the server again.

[QUOTE=jyb;240789]I think I must be misinterpreting your proposal because I don't see how it accomplishes either of those aims. The client still performs exactly the work that the server sends it, right? That work is now a longer list of numbers/curves than before, but in what way does the end user have more control?

And how does this relate to reservations? Is the idea that once the server sends work to one client, it is reserved (and therefore not sent to anyone else) until either the work is sent back as either released or completed, or the client times out? It's not clear that that would be a desirable change to make (I could probably be convinced), but either way it seems orthogonal to the proposal you've made (unless, again, I'm just not correctly interpreting your proposal).[/QUOTE]

With my proposal multiple clients could work on the same number concurrently. By doing this, ECM could be completed much faster for candidate numbers (depending upon configuration). The server admin would have more visibility into which clients are connected and the work they are doing. There is the possibility of multiple clients returning the same factor, but that exists today as well. With today's server there is no tracking of reservations, just a timestamp for when a candidate number was sent to a client. If the client doesn't respond, various rules are used to determine when that number can be sent to another client. By limiting the amount of work sent to each client (which is not done today), the server has a better chance of getting a response before expiring a reservation.

On the client side, I want to make it easier for a client to bounce between numbers to factor. Instead of having to do all remaining ECM curves at a given B1 for a given number (until timer expires or a factor is found), the client could do a smaller number of curves for each number.

[QUOTE=rogue;240803]
[QUOTE=jyb;240789]What is the "it" in that sentence? The server? I.e. are you just proposing that rather than sending a single number with a certain number of curves you send a somewhat longer list of work units to perform?
[/QUOTE]
Yes. If the server has lots of small numbers to run curves on, then the client could attempt to factor more than one before communicating with the server again.
[/QUOTE]

Okay, I guess I see how this gives the client a little more control. But see below.

[QUOTE=rogue;240803]
With my proposal multiple clients could work on the same number concurrently. By doing this, ECM could be completed much faster for candidate numbers (depending upon configuration). The server admin would have more visibility into which clients are connected and the work they are doing.
[/QUOTE]

So maybe here's where I'm just not getting it, and I hope you can clear it up for me. It is already the case today that multiple clients can work on the same number concurrently. So in what way are you envisioning that ECM would get completed faster or that the server admin would have more visibility into anything?

[QUOTE=rogue;240803]There is the possibility of multiple clients returning the same factor, but that exists today as well. With today's server there is no tracking of reservations, just a timestamp for when a candidate number was sent to a client. If the client doesn't respond, various rules are used to determine when that number can be sent to another client. By limiting the amount of work sent to each client (which is not done today), the server has a better chance of getting a response before expiring a reservation.[/QUOTE]

But the amount of work [B][I]is[/I][/B] limited today. It's limited to the number of curves remaining for the given number at the given B1 level. Or do you mean that it would limit it further, based on other clients already having "reserved" some curves?

[QUOTE=rogue;240803]On the client side, I want to make it easier for a client to bounce between numbers to factor. Instead of having to do all remaining ECM curves at a given B1 for a given number (until timer expires or a factor is found), the client could do a smaller number of curves for each number.[/QUOTE]

But the client can always do a smaller number of curves than the number given right now, by simply setting its timer short enough. And when the B1 level is at all high, then this is always what happens in practice anyway, except when all numbers being handed out are almost finished at the current B1 level. So if a client wants to bounce around, it can set its timer to a value that reflects the number of curves it wants to do before bouncing.

[QUOTE=rogue;239671]I'm working again on the changes to ECMNet. I have a question for those of you who set up servers.

Does anyone use master/slave configurations?[/QUOTE]I use them. I like to have machines working on a uniform set of candidates for ease of administration. Some of the machines do not have direct internet accessibility but do have connectivity to a gateway system. Running a slave server on the gateway allows them to participate in a uniform project.

Paul

[QUOTE=rogue;239671]Does anyone use master/slave configurations?[/QUOTE][AOL]Me too![/AOL]

I set my ECM group up with one master and then a slave on each client machine. That way the clients will never run out of work should the network or the master server go down (usually due to the power getting interuppted, although I did have a WAP go down which cut my netowrk in half).

Does anyone modify the localcontrol or recurse settings? I'm wondering if I need to continue supporting these?

I'm looking for source code (in C or C++) for a relatively simple expression parser. Does anyone know where I could find one? I would consider PFGW's, but it's expression parser is complex and rather nasty. I haven't looked at GMP-ECM's, so I don't know if that one would be feasible to use.

Here is a brief update.

Things are going slowly. I have a table creation script, which I think it complete, but I won't know until I get further into development. Right now I'm focusing on master/slave synchronization. The rewrite for this is much more complex due to the databases as both master and slave will have to manage transactions. I'm expecting to take three days of coding just for this piece. I think finishing this release will take a lot longer than I anticipated.

[QUOTE=rogue;241788]Does anyone modify the localcontrol or recurse settings? I'm wondering if I need to continue supporting these?[/QUOTE]I've never used them.

Paul

[QUOTE=xilman;241963]I've never used them.[/QUOTE]

Thanks.

Here is another update.

I finished the master/slave coding a couple of weeks ago. Right now a master/slave configuration is expecting all servers to be at version 3 otherwise they won't sync. You'll be able to upgrade them independently, but syncing won't be done until both master and slave are both at v3. The problem with supporting a mix of v2 and v3 is that v2 has no transaction boundaries so it is impossible to know if syncing works correctly. The alternative (which would be nasty to code) would be to handle the v3 side as a single transaction, which would lock up any clients trying to connect to either server for an inordinate amount of time.

I have switched over to working on the client side. The message format in v3 will be completely different than in v2. The server will send a single message for each composite to factor and expect one per composite work done and one per factor. Fortunately I will maintain (until version 4.x, if there ever is one) backward compatibility so a v3 client will work with a v2 server and a v2 client will work with a v3 server.

I have also started working on the changes to the .ini files (the .cfg files will have a new extension since ecmserver.ini will be gone), including the -u option to load the database from the ecmserver.ini file (which needs to be renamed before upgrading). The b1 limits will now be in the database instead of a file. They will be inserted as part of the install script, but can be changed on the fly, whereas previously changes to the b1 limits required a shutdown/restart.

Unfortunately I have been too busy to continue working on it. I hope to have some time in the evenings, but I cannot make any promises.

As for an expression parser, I will probably steal eval.c from GMP-ECM for ECMNet's use. I'll just have to ask Paul Zimmerman if he would be okay with that. I haven't look in depth at it, but hopefully it won't be too embedded in GMP-ECM.

Another update.

The coding is nearing completion. With permission, I borrowed eval.c from GMP-ECM and that code is now integrated into the server. I have written all of the code regarding getting and returning work and factors. I have also finished the code behind user stats. I believe that the only remaining code is for handling the addition of new composites and factors via ecmadmin.

I've decided to keep the b1 limits in the ini file. I started doing it in the database, but then realized that I would need to select them frequently and that could hurt server performance. If anyone has any heartburn due to my not keeping the b1 limits in the database, please let me know.

After talking with yoyo, I would like to BOINCify the client. I see no reason why the project couldn't build a BOINC version and a non-BOINC version of the client. It might not be quite so easy to do that, but once done would make it easier for the BOINC version to stay in sync with the non-BOINC version. The two of us will have to decide what is best going forward. Those changes might not make it into 3.0.

With the new ECMNet, the server is collecting additional statistical information (because we know how people like statistics).

Presuming I'm not too wiped tomorrow night, I will finish up the coding and start testing.

FWIW, BOINC-ification of the ggnfs code for RSALS was a matter of several dozens SLOC for calling the appropriate BOINC API functions at the appropriate places.
NFS@Home's BOINC-ified ggnfs may be slightly different from that of RSALS, as Greg Childers uses a different parameter passing scheme. I don't remember. But I'm positive that both diffs against upstream are small.

FYI, I finished coding last week. I had intended to finish up earlier, but I had some late nights and didn't have the energy to work on it. I have yet to do any testing. If I can find the time I might be able to get started next week.

[QUOTE=rogue;254018]FYI, I finished coding last week. I had intended to finish up earlier, but I had some late nights and didn't have the energy to work on it. I have yet to do any testing. If I can find the time I might be able to get started next week.[/QUOTE]If I can be of assistance in testing, please ask.

Paul

[QUOTE=xilman;254067]If I can be of assistance in testing, please ask.

Paul[/QUOTE]

Thanks. I will look for testers once I can get it to beta. I haven't run the software yet. I have only built it.

Where can I find the old ecmserver client binaries for windows?

Thank you in advance.

[QUOTE=em99010pepe;256943]Where can I find the old ecmserver client binaries for windows?

Thank you in advance.[/QUOTE]

I presume you mean 2.7.x. You can get them [URL="http://home.roadrunner.com/~mrodenkirch/home/ECMNet.html"]here[/URL].

[QUOTE=rogue;256952]I presume you mean 2.7.x. You can get them [URL="http://home.roadrunner.com/%7Emrodenkirch/home/ECMNet.html"]here[/URL].[/QUOTE]

The binaries?

[QUOTE=em99010pepe;256959]The binaries?[/QUOTE]

I'm sorry. I hadn't realized that I never included the Windows binaries with the source. d/l again. I just updated the zip file.

I would like to start some testing of ECMNet 3.0. One of the tests I need to run is the upgrade from an ECMNet 2.x ecmserver.ini file. If you are running a server, would you mind e-mailing it to me?

[QUOTE=rogue;257071]I would like to start some testing of ECMNet 3.0. One of the tests I need to run is the upgrade from an ECMNet 2.x ecmserver.ini file. If you are running a server, would you mind e-mailing it to me?[/QUOTE]

Sorry but I am still not running the ecmnet server.

[QUOTE=rogue;257071]I would like to start some testing of ECMNet 3.0. One of the tests I need to run is the upgrade from an ECMNet 2.x ecmserver.ini file. If you are running a server, would you mind e-mailing it to me?[/QUOTE]

Thanks guys. I have received 5 different files. I'll do some testing tomorrow.

I'm pleased to relate that all ini files (even Paul's which has over 12000 composites) loaded flawlessly, at least as far as I can tell. I did run into a couple of bugs as part of the import, but those are fixed. I did discover a couple of problems in one ini file, which could be a problem in the 2.x code that I am unaware of.

The next testing I need to do is on client/server communication. Maybe I can start on that this weekend.

[QUOTE=rogue;257285]I'm pleased to relate that all ini files (even Paul's which has over 12000 composites) loaded flawlessly, at least as far as I can tell. [/quote]Good!
[QUOTE=rogue;257285]
The next testing I need to do is on client/server communication. Maybe I can start on that this weekend.[/QUOTE]If you would like me to run test clients and/or servers, please ask.

Paul

[QUOTE=xilman;257289]Good!
If you would like me to run test clients and/or servers, please ask.

Paul[/QUOTE]

Thanks.

Eventually I will initiate some alpha testing and I could use some help then, but before then I need to make sure that the most egregious errors in the code are fixed. Since the protocol has been completely rewritten, there is a bit of testing to be done. These are some of the areas that need testing and the order in which I intend to attack them:

1) ECMNet 3.0 server working with ECMNet 3.0 client.
2) ECMNet 3.0 server working with ECMNet 2.x client.
3) ECMNet 2.x server working with ECMNet 3.0 client.
4) Factor submission and e-mail notification.
5) Master/slave (3.0 server to 3.0 server only).

As stated previously, I have no intention to make the master/slave logic be backward compatible with 2.x servers.

I have been fortunate to have a lot of time to test v3.0 in the past couple of days. The testing is going slowly, but I do have the client and server talking to one another along with the appropriate database updates. I haven't tested the code for newly found factors yet, but hopefully I will get to it this week. The interoperability of 2.x and 3.0 is really an issue of 3.0 using the correct protocol. That should go fairly quickly.

I have removed the logic to send multiple types of factoring work at one time. It was proving to be too complex and I didn't want the 3.0 release to be dependent upon it working. Fortunately the 3.0 client should support it, if I do ever put the time into getting it to work in the server.

I revamped the server_stats.html page to not show all of the details for each composite. It will show the current B1 and number of factors found for the composite, but it won't show factoring work done or the factors themselves. So that the functionality isn't lost, I have added a composites.html page which closely corresponds to the old server_stats.html page. The nice thing about this change is that server_stats.html will take a lot less time to generate than composites.html since they both need to hit the database to generate the data.

I have added a user_stats.html page which shows how much work and time was spent by each user (and how many factors they found). I'm suspecting that some stats lovers will gravitate towards ECMNet if they have a means of seeing how much work they have done over time.

All in all, once I feel that the 3.0 server and 3.0 client are working well together, I might release an alpha before testing either with its 2.x counterpart. It would be for those who have a private ECMNet server who are willing to subject themselves to an alpha test.

[QUOTE=rogue;257617]All in all, once I feel that the 3.0 server and 3.0 client are working well together, I might release an alpha before testing either with its 2.x counterpart. It would be for those who have a private ECMNet server who are willing to subject themselves to an alpha test.[/QUOTE]Count me in.

Paul

I have added a "pending_work.html" webpage to the server. This will show the outstanding, i.e. not returned, factoring work sent to clients.

Note that this is not a reservation system. It is only intended to give people an idea of how many clients are actively doing factoring work for that server. There is no requirement that returned work marries up to one of the "pending work" records.

Via configuration, the admin can control how long they want records to show up on this page, thus presuming that a client has abandoned work and won't be returning it.

A change I made to the client will allow the user to control what they want to do when shutting down. Currently the client automatically returns completed work to the server. The change is to allow the client to shutdown without returning that work so that they can restart and continue with the previous factoring assignment. Grabbed from PRPNet, this is called "stopoption" in the ini file.

There is also a change in the client to allow the client to shutdown upon completion of a curve. Also grabbed from PRPNet, this is called "stopasaption" in the ini file. The ini file is read after each curve is completed. By looking at stopasapoption, the client can be told to report the completed work and shutdown as soon as it has completed working on the current curve.

[QUOTE=rogue;257851]I have added a "pending_work.html" webpage to the server. This will show the outstanding, i.e. not returned, factoring work sent to clients.

Note that this is not a reservation system. It is only intended to give people an idea of how many clients are actively doing factoring work for that server. There is no requirement that returned work marries up to one of the "pending work" records.

Via configuration, the admin can control how long they want records to show up on this page, thus presuming that a client has abandoned work and won't be returning it.
[/QUOTE]

I assume that goes both ways? I.e. there may never be returned work for some given outstanding work, but is it also the case that work can be returned which the server doesn't have in its pending work records?


[QUOTE=rogue;257851]
A change I made to the client will allow the user to control what they want to do when shutting down. Currently the client automatically returns completed work to the server. The change is to allow the client to shutdown without returning that work so that they can restart and continue with the previous factoring assignment. Grabbed from PRPNet, this is called "stopoption" in the ini file.

There is also a change in the client to allow the client to shutdown upon completion of a curve. Also grabbed from PRPNet, this is called "stopasaption" in the ini file. The ini file is read after each curve is completed. By looking at stopasapoption, the client can be told to report the completed work and shutdown as soon as it has completed working on the current curve.[/QUOTE]

That last one is identical to the current behavior on receipt of a SIGTERM, right?

[QUOTE=jyb;257873]I assume that goes both ways? I.e. there may never be returned work for some given outstanding work, but is it also the case that work can be returned which the server doesn't have in its pending work records?

That last one is identical to the current behavior on receipt of a SIGTERM, right?[/QUOTE]

Yes to your first question.

No. In the ECMNet client SIGTERM immediately stops working on the curve. I don't know what GMP-ECM does (as the SIGTERM will be passed to it as well), by the client acts as if GMP-ECM was terminated without completng a curve.

The difference is that today the client will always report completed work upon ^C, i.e. the user has no choice in the matter. stopoption gives the user the choice of what to do with completed work with ^C is hit, i.e. return or hold on to it to do more work on it later.

stopasapoption does not deal with signal handling at all. If GMP-ECM is running and the user decides that they want to complete the current curve run by GMP-ECM (without interrupting GMP-ECM), then shut down, they would update this option in the ini file. The client will read the ini file after each curve is done and decide if it needs to shut down and upon shut down if the completed work is to be returned or held (like stopoption).

I hope that clears up any confusion.

[QUOTE=rogue;257876]Yes to your first question.

No. In the ECMNet client SIGTERM immediately stops working on the curve. I don't know what GMP-ECM does (as the SIGTERM will be passed to it as well), by the client acts as if GMP-ECM was terminated without completng a curve.

The difference is that today the client will always report completed work upon ^C, i.e. the user has no choice in the matter. stopoption gives the user the choice of what to do with completed work with ^C is hit, i.e. return or hold on to it to do more work on it later.

stopasapoption does not deal with signal handling at all. If GMP-ECM is running and the user decides that they want to complete the current curve run by GMP-ECM (without interrupting GMP-ECM), then shut down, they would update this option in the ini file. The client will read the ini file after each curve is done and decide if it needs to shut down and upon shut down if the completed work is to be returned or held (like stopoption).

I hope that clears up any confusion.[/QUOTE]

I didn't ask about hitting ^C, I asked about sending a SIGTERM to the ecmclient process. Hitting ^C causes a SIGINT to be sent to all processes in the foreground process group, which would ordinarily include the GMP-ECM process as you say, but that's not what I'm talking about.

This has already been covered earlier in this thread. See post #24. As far as I can tell, sending a SIGTERM (or SIGINT) to *just the ecmclient process* has exactly the behavior you're going for with stopasapoption.

[QUOTE=jyb;257878]I didn't ask about hitting ^C, I asked about sending a SIGTERM to the ecmclient process. Hitting ^C causes a SIGINT to be sent to all processes in the foreground process group, which would ordinarily include the GMP-ECM process as you say, but that's not what I'm talking about.

This has already been covered earlier in this thread. See post #24. As far as I can tell, sending a SIGTERM (or SIGINT) to *just the ecmclient process* has exactly the behavior you're going for with stopasapoption.[/QUOTE]

That might be true on *nix (I'm not an expert on *nix signals), but not on Windows . Remember that this code also has to run on Windows and Windows doesn't have *nix signal handling.

[QUOTE=rogue;257880]That might be true on *nix (I'm not an expert on *nix signals), but not on Windows . Remember that this code also has to run on Windows and Windows doesn't have *nix signal handling.[/QUOTE]

Fair enough. I can't claim any expertise on (or even passing knowledge of) how these things work in Windows. Is there really no way to do the equivalent of sending a signal to just the ecmclient process? My reading of the code, even for Windows, is that ecmclient behaves exactly the same way as I described. It's just a question of what the ecm process will do.

[QUOTE=jyb;257883]Fair enough. I can't claim any expertise on (or even passing knowledge of) how these things work in Windows. Is there really no way to do the equivalent of sending a signal to just the ecmclient process? My reading of the code, even for Windows, is that ecmclient behaves exactly the same way as I described. It's just a question of what the ecm process will do.[/QUOTE]

Windows uses a "control handler", which is similar to *nix signal handler. In code you specify a routine to get control when a user does something, such as closing the window of the parent process or killing via the Task Manager. I don't differentiate between the different types of events in code.

To me the important thing is that the software behaves in a simple, predictable fashion and that a user has some ability to control the software's behavior without understanding the code behind it. I won't preclude anyone from using SIGTERM, but I doubt that it would be the preferred method for users to shutdown the client. FYI, out of habit I never rely on software to handle SIGTERM cleanly. Maybe I'm paranoid, but that is me. I do rely on software to handle SIGINT correctly.

I was merrily testing today when I realized that I hadn't written a piece of code that is needed: the recursive factoring of composites co-factors. I wrote the code, but then I wondered if I designed the tables optimally. I pose a question to you, the users of the ECMNet server to get you opinions.

Should I treat the new database similar to the current ini file, where connection between composite co-factor and original composite is lost (if you don't consider that the prefix of the composite co-factor name is the same as the original composite name) or should I continue to track the original composite and treat each factor as a child to that composite?

This is the first option would look like in the database:

[code]
composite a
factor a-1
factor a-2

composite a-1
factor a-1-1
factor a-1-2

composite a-1-2
factor a-1-2-1
factor a-1-2-2

...
[/code]

This is the second option would look like in the database:

[code]
composite a
factor a-1
factor a-2
factor a-3
...
factor a-n
[/code]

Both have distinct advantages. The first is similar to the 2.x design. The second allows one to easily see all factors for a composite. The first requires the server to track work for all composites and their composite co-factors. The second requires the server to track work only for the original composite.

I'm personally leaning toward the second option because it makes more logical sense.

Opinions?

I seldom use automatic recursive factorization, so my opinion should be given low weight. I prefer option 2.

William

[QUOTE=wblipp;258055]I seldom use automatic recursive factorization, so my opinion should be given low weight. I prefer option 2.

William[/QUOTE]I always use automatic recursive factorization. Given that my scripts for keeping things tidy are going to have to change anyway, it doesn't really matter to me which option you choose. The second looks more aesthetically pleasing.

Paul

Okay, that is what I will do. It will cost me some time and delay the release.

Does anyone care about "difficulty"? The difficulty is computed as the total work done * (decimal length^2). You can set up a strategy to order work by difficulty, with less difficult numbers to be factored before more difficult numbers.

Note that there is already a means to order work selection by total work and one to order work selected by decimal length. This would be a carry over from 2.x. There will be no means to order work selection by b1, as it is presumed that total work should be sufficient. Total work is computed by sum(b1 * ECM curves for b1) for all b1.

Opinions?

[QUOTE=rogue;258182]Does anyone care about "difficulty"? The difficulty is computed as the total work done * (decimal length^2). You can set up a strategy to order work by difficulty, with less difficult numbers to be factored before more difficult numbers.[/QUOTE]

I use difficulty a lot - usually 60-90% in the configuration. I like the resulting spread of much more work for smaller numbers.

[QUOTE=rogue;258182]Does anyone care about "difficulty"? The difficulty is computed as the total work done * (decimal length^2). You can set up a strategy to order work by difficulty, with less difficult numbers to be factored before more difficult numbers.[/QUOTE]To the best of my knowledge, I've never ordered by "difficulty". Below is the portion of the current ecmserver.cfg if that helps you understand my approach.
[code]// strategy0 - random
// strategy1 - by lowest B1 that needs curves done and total work done
// strategy2 - by total work done
// strategy3 - by difficulty
// strategy4 - by length and total work done
//strategy0=20:20:0:0:0
strategy0=00:20:0:0:0
//strategy1=80:20:0:0:0
strategy1=100:20:0:0:0
strategy2=0:20:0:0:0
strategy3=0:20:0:0:0
strategy4=0:20:0:0:0[/code]The two commented lines are my other standard approach and are left there so that changing between the alternatives doesn't require any thinking and, more important, less likely to incur typos.

To be honest, I've never used any complex strategies during my entire experience of running your code. The last three have always had 0:20:0:0:0 and the last three figures on every line have never been anything other than 0:0:0. I'm a bear of very little brain and those things confuse me.

Paul


Paul

[QUOTE=rogue;258179]Okay, that is what I will do. It will cost me some time and delay the release.[/QUOTE]Thanks, but please don't let my tastes delay a release. I'm happy to wait for recursive factorization on the assumption that it won't be too difficult for me to implement recursion by manual modifications to the database.

Paul

[QUOTE=rogue;258182]There will be no means to order work selection by b1, as it is presumed that total work should be sufficient. Total work is computed by sum(b1 * ECM curves for b1) for all b1.[/QUOTE]I don't think I understand this. Does it mean that you have removed the old[code]// strategy1 - by lowest B1 that needs curves done and total work done[/code] functionality? If so, I'll have to rethink my present choices. Or does your quoted material restate the old explanation for strategy1?


Paul

[QUOTE=xilman;258194]Thanks, but please don't let my tastes delay a release. I'm happy to wait for recursive factorization on the assumption that it won't be too difficult for me to implement recursion by manual modifications to the database.

Paul[/QUOTE]

Recursion is there. It was a question of whether or not to remove the ability to turn it off.

[QUOTE=xilman;258195]I don't think I understand this. Does it mean that you have removed the old[code]// strategy1 - by lowest B1 that needs curves done and total work done[/code] functionality? If so, I'll have to rethink my present choices. Or does your quoted material restate the old explanation for strategy1?

Paul[/QUOTE]

strategy1 and strategy2 are so similar that one could argue that they would produce the same results. Based upon the total work done, the server will determine the current B1 that needs work and how many curves are needed for that B1.

I'll try to explain how it works and hopefully that will allay any fears you may have about losing functionality.

When the server starts up, it reads in ecmserver.b1 and builds a table, which looks something like this:
[code]
B1 Curves Total Work
300 8 308
2000 30 62400
11000 74 874000
50000 221 11864000
250000 453 124300000
1000000 984 1097250000
3000000 2541 8607000000
11000000 4949 62062000000
43000000 8266 409877000000
110000000 20158 2572818000000
260000000 47173 14482360000000
850000000 77666 78281080000000
2900000000 96000 344416100000000
9900000000 445000 4683900000000000[/code]

The total ECM work done for each composite is stored in a column (don't worry the actual b1s and curves run are still there as well). When the work selection routine is called, it uses that value and does a lookup in the table above to find the current b1 level needing work. For example, if the total work for a composite is 2000000000, then it indicates that all curves for b1 <= 1e6 are probably done and that some curves for 3e6 are probably done. It then does a computation (8607000000 - 2000000000) / 3e6 and computes that approximately 2202 curves for b1=3e6 needs to be run.

So unless higher b1 for ECM are run out of sequence, this should provide the same results for both strategy1 and strategy2.

If you still have concerns, please let me know.

I finally finished with the latest set of code changes. Now back to testing.

With the holiday weekend I have had no time to test and next weekend will be just as bad because I will be on vacation for the next few days with no access to a computer or phone (as vacations should be).

I finally got back to some testing this weekend. Things are looking good so far, but I have a lot more testing to do before I can release an alpha.