ECMNet Feature Requests

[rogue]

I'm working again on the changes to ECMNet. I have a question for those of you who set up servers.

Does anyone use master/slave configurations?

In the PRPNet world, I don't support it because clients can d/l as much work as the server allows. While the client has work it is reserved. This means that no other client will get a PRP test that is reserved by another client.

In the ECMNet world, things are different. It is common to have multiple clients doing ECM curves on the same number at the same time. This is controlled via the "hour breakpoints" in the ecmserver.cfg. The server also does not track reservations as the client will continue doing curves on the number until it reaches the max. This means that a client could have outstanding work for days, weeks, or longer.

I have considered some changes so that the client could get workunits and then report back to the server when the workunits have completed or when a certain amount of work has been done. If I do this, then I could set up a reservation system which could also be used to vary the work a little more with the clients. For example, I could have it send this:

Type=P+1 Curves=3 B1=100000 Number=Number1
Type=P-1 Curves=1 B1=100000 Number=Number1
Type=ECM Curves=100 B1=100000 Number=Number1
Type=P+1 Curves=3 B1=100000 Number=Number2
Type=P-1 Curves=1 B1=100000 Number=Number2
Type=ECM Curves=100 B1=100000 Number=Number2
Type=P+1 Curves=3 B1=300000 Number=Number2
Type=P-1 Curves=1 B1=300000 Number=Number2
Type=ECM Curves=100 B1=300000 Number=Number2

etc.

until max work per client is exceeded.

The number of curves is evaluated as MIN(curves left for that B1, max per client for that B1). The client would return completed work when either:
a) all work completed
b) max time for server exceeded (for example, client does curves for 10 hours, then returns what was completed, the max time is configurable)
c) max work exceeded (work is a computed value based upon decimal length of the number and B1, when a curve is completed, work is incremented, max work is a configurable threshold).

I'm not proposing to get rid of the "hour breakpoints", but I want to allow the end user to have more control over the work they do and the allow the server admin more control over reservations.

Thoughts?

[jyb]

Originally Posted by rogue

[snip]

For example, I could have it send this:

What is the "it" in that sentence? The server? I.e. are you just proposing that rather than sending a single number with a certain number of curves you send a somewhat longer list of work units to perform?

Originally Posted by rogue

I'm not proposing to get rid of the "hour breakpoints", but I want to allow the end user to have more control over the work they do and the allow the server admin more control over reservations.

Thoughts?

I think I must be misinterpreting your proposal because I don't see how it accomplishes either of those aims. The client still performs exactly the work that the server sends it, right? That work is now a longer list of numbers/curves than before, but in what way does the end user have more control?

And how does this relate to reservations? Is the idea that once the server sends work to one client, it is reserved (and therefore not sent to anyone else) until either the work is sent back as either released or completed, or the client times out? It's not clear that that would be a desirable change to make (I could probably be convinced), but either way it seems orthogonal to the proposal you've made (unless, again, I'm just not correctly interpreting your proposal).

[wblipp]

Originally Posted by rogue

Does anyone use master/slave configurations?

I use them.

I use them to segregate work units in various ways. Odd Perfect Number has a server that hands out only roadblocks and another that hands out only small tasks. Users can prioritize based on interest and computer power.

While I was living in London, I used a slave server for administration of the OPN servers that stayed in the USA. I could easily get an updated ecmserver.ini, and I also reported factors to the master by hand creating factor lines in the slave's ecmserver.ini.

William

[rogue]

Originally Posted by jyb

What is the "it" in that sentence? The server? I.e. are you just proposing that rather than sending a single number with a certain number of curves you send a somewhat longer list of work units to perform?

Yes. If the server has lots of small numbers to run curves on, then the client could attempt to factor more than one before communicating with the server again.

Originally Posted by jyb

I think I must be misinterpreting your proposal because I don't see how it accomplishes either of those aims. The client still performs exactly the work that the server sends it, right? That work is now a longer list of numbers/curves than before, but in what way does the end user have more control?

And how does this relate to reservations? Is the idea that once the server sends work to one client, it is reserved (and therefore not sent to anyone else) until either the work is sent back as either released or completed, or the client times out? It's not clear that that would be a desirable change to make (I could probably be convinced), but either way it seems orthogonal to the proposal you've made (unless, again, I'm just not correctly interpreting your proposal).

With my proposal multiple clients could work on the same number concurrently. By doing this, ECM could be completed much faster for candidate numbers (depending upon configuration). The server admin would have more visibility into which clients are connected and the work they are doing. There is the possibility of multiple clients returning the same factor, but that exists today as well. With today's server there is no tracking of reservations, just a timestamp for when a candidate number was sent to a client. If the client doesn't respond, various rules are used to determine when that number can be sent to another client. By limiting the amount of work sent to each client (which is not done today), the server has a better chance of getting a response before expiring a reservation.

On the client side, I want to make it easier for a client to bounce between numbers to factor. Instead of having to do all remaining ECM curves at a given B1 for a given number (until timer expires or a factor is found), the client could do a smaller number of curves for each number.

[jyb]

Originally Posted by rogue

Yes. If the server has lots of small numbers to run curves on, then the client could attempt to factor more than one before communicating with the server again.

Okay, I guess I see how this gives the client a little more control. But see below.

Originally Posted by rogue

With my proposal multiple clients could work on the same number concurrently. By doing this, ECM could be completed much faster for candidate numbers (depending upon configuration). The server admin would have more visibility into which clients are connected and the work they are doing.

So maybe here's where I'm just not getting it, and I hope you can clear it up for me. It is already the case today that multiple clients can work on the same number concurrently. So in what way are you envisioning that ECM would get completed faster or that the server admin would have more visibility into anything?

Originally Posted by rogue

There is the possibility of multiple clients returning the same factor, but that exists today as well. With today's server there is no tracking of reservations, just a timestamp for when a candidate number was sent to a client. If the client doesn't respond, various rules are used to determine when that number can be sent to another client. By limiting the amount of work sent to each client (which is not done today), the server has a better chance of getting a response before expiring a reservation.

But the amount of work is limited today. It's limited to the number of curves remaining for the given number at the given B1 level. Or do you mean that it would limit it further, based on other clients already having "reserved" some curves?

Originally Posted by rogue

On the client side, I want to make it easier for a client to bounce between numbers to factor. Instead of having to do all remaining ECM curves at a given B1 for a given number (until timer expires or a factor is found), the client could do a smaller number of curves for each number.

But the client can always do a smaller number of curves than the number given right now, by simply setting its timer short enough. And when the B1 level is at all high, then this is always what happens in practice anyway, except when all numbers being handed out are almost finished at the current B1 level. So if a client wants to bounce around, it can set its timer to a value that reflects the number of curves it wants to do before bouncing.

[xilman]

Originally Posted by rogue

I'm working again on the changes to ECMNet. I have a question for those of you who set up servers.

Does anyone use master/slave configurations?

I use them. I like to have machines working on a uniform set of candidates for ease of administration. Some of the machines do not have direct internet accessibility but do have connectivity to a gateway system. Running a slave server on the gateway allows them to participate in a uniform project.

Paul

[schickel]

Originally Posted by rogue

Does anyone use master/slave configurations?

[AOL]Me too![/AOL]

I set my ECM group up with one master and then a slave on each client machine. That way the clients will never run out of work should the network or the master server go down (usually due to the power getting interuppted, although I did have a WAP go down which cut my netowrk in half).

[rogue]

Does anyone modify the localcontrol or recurse settings? I'm wondering if I need to continue supporting these?

[rogue]

I'm looking for source code (in C or C++) for a relatively simple expression parser. Does anyone know where I could find one? I would consider PFGW's, but it's expression parser is complex and rather nasty. I haven't looked at GMP-ECM's, so I don't know if that one would be feasible to use.

[rogue]

Here is a brief update.

Things are going slowly. I have a table creation script, which I think it complete, but I won't know until I get further into development. Right now I'm focusing on master/slave synchronization. The rewrite for this is much more complex due to the databases as both master and slave will have to manage transactions. I'm expecting to take three days of coding just for this piece. I think finishing this release will take a lot longer than I anticipated.

[xilman]

Originally Posted by rogue

Does anyone modify the localcontrol or recurse settings? I'm wondering if I need to continue supporting these?

I've never used them.

Paul