I thought the 3000 ID/Hour limit was for unregistered users only!

[Stargate38]

Why do I have a limit of 3000 IDs/Hour? I thought that was for unregistered users only, not registered users! My DB Name is Daniel if you want to know. Please fix that. I want to do Aliquot sequences and I don't want to be stuck doing half the sequence, then waiting an hour to post the rest! Please at least triple it!

Also, Why can't I get the DB Help page (question mark link) to work when I exceed the limit? That page should work. The "Imprint" link works no matter what limit is exceeded (I found that out last month).

Please! It makes it hard to use AliWin2 and Wget.

[fivemack]

Originally Posted by Stargate38

Why do I have a limit of 3000 IDs/Hour? I thought that was for unregistered users only, not registered users! My DB Name is Daniel if you want to know. Please fix that. I want to do Aliquot sequences and I don't want to be stuck doing half the sequence, then waiting an hour to post the rest! Please at least triple it!

I too do aliquot sequences. In three CPU-years I have accumulated around 19000 lines; for it to require six attempts over an afternoon to post the results of three CPU-years seems not unreasonable.

If you are doing trivial aliquot sequences such that the wait to deposit the results in the database is a perceptible fraction of the compute time, I would accuse you of polluting the database in exactly the way that the limit on IDs per hour was intended to avoid, and strongly suggest that you set your computers on meatier targets.

[schickel]

Originally Posted by Stargate38

Why do I have a limit of 3000 IDs/Hour? I thought that was for unregistered users only, not registered users!

Yes, if you are logged in, you get a little more in the way of IDs an hour, but that's only if you report results using your credentials. You would have to check with EdH, but it sounds like AliWin and wget are probably not using credentials when turning in results....

My DB Name is Daniel if you want to know. Please fix that. I want to do Aliquot sequences and I don't want to be stuck doing half the sequence, then waiting an hour to post the rest! Please at least triple it!

Also, Why can't I get the DB Help page (question mark link) to work when I exceed the limit? That page should work.

You're right, Syd made the blocking a little wide, but there were some serious spamming issues that he had to combat. I found out the hard way that it can be a little difficult to deal with one time when I forgot to log in before dumping in some results.....but as mentioned, the spam he is blocking was worse that the little inconveniences we might have to put up with....

The "Imprint" link works no matter what limit is exceeded (I found that out last month).

Please! It makes it hard to use AliWin2 and Wget.

As I said, I think the problem is with AliWin and wget.

I also have an idea that if you're doing a lot of work on sequences starting in the range >1M, resulting in lots of new lines with factors in the mid-size range, you're going to run into this issue more than someone who turns in something more along the lines of a couple of hundred lines with lots of already-existing small to medium primes and just a few large new primes......

[Mini-Geek]

Originally Posted by fivemack

I too do aliquot sequences. In three CPU-years I have accumulated around 19000 lines; for it to require six attempts over an afternoon to post the results of three CPU-years seems not unreasonable.

Nitpicking: X lines != X IDs. X previously-unknown primes would be closer to X IDs (assuming that in a batch of submissions the DB doesn't create IDs for composites; otherwise it gets more complicated and goes up much higher), and you surely average more than one of those per line. I think if someone added 500 lines on normal sized aliquot sequence(s) all at once, you could hit 3000 pretty easily. Unless you have a lot of hardware or can not easily set up AliWin/wget to automatically submit the results periodically, this shouldn't bother you.

[EdH]

AliWin2 does submit periodically (about every two hours) if "Auto Send" is checked. (I do have some more work to do in that area, so only new lines are sent and if none exist, the send is skipped.)

I have no knowledge of how to send credentials with the wget/Aliqueit method. AliWin2 simply invokes the Aliqueit method of:

aliqueit -s 0 ######

If someone could tell me the how to add credentials, I'll change the AliWin2 code to submit in that manner.

[wblipp]

Originally Posted by EdH

If someone could tell me the how to add credentials, I'll change the AliWin2 code to submit in that manner.

Try logging into factordb using both IE and firefox. I have had problems in the past with auto-connection tools taking cookies from a browser I did not normally use - perhaps that is going on here.

[Dubslow]

Originally Posted by wblipp

Try logging into factordb using both IE and firefox. I have had problems in the past with auto-connection tools taking cookies from a browser I did not normally use - perhaps that is going on here.

Python and Perl maintain their own cookies -- presumably the networking libraries for C/C++ also allow you to do the same. Unfortunately, I have no clue if that's true or not, or how to do so. If no one volunteers to, e.g. modify aliqueit, aliwin, etc., I could put together a python script that submits with credentials (assuming of course that FDB maintains the login session in the cookie). That would require aliwin modification anyways, but at least you wouldn't have to figure out the networking libraries. It's up to you.

Edit: Prime95 uses the cURL library (libcurl), though admittedly, Prime95 doesn't bother with cookies, the username is sent as part of each request. Wget apparently does support cookies (http://en.wikipedia.org/wiki/Wget#Advanced_examples), though I don't actually know the first thing about using it.

(From the link above)
# Using wget to download content protected by referer and cookies.
# 1. get base url and save its cookies in file
# 2. get protected content using stored cookies
wget --cookies=on --keep-session-cookies --save-cookies=cookie.txt http://first_page
wget --referer=http://first_page --cookies=on --load-cookies=cookie.txt --keep-session-cookies --save-cookies=cookie.txt http://second_page

[EdH]

Originally Posted by wblipp

Try logging into factordb using both IE and firefox. I have had problems in the past with auto-connection tools taking cookies from a browser I did not normally use - perhaps that is going on here.

Sorry, I'm still a little confused. Are you saying that logging in to the db via a browser, which will set a cookie, and then using the aliqueit -s option, will allow the data to be accepted as from a logged in user?

Or, is there a way to send login info along with the factors via wget?

AliWin2 simply uses wget via the -s option in Aliqueit to submit lines from the elf, currently. I make no attempt to log into the db before using the -s option. And, I don't know how to tell if lines submitted were credited to my login.

Thanks.

@Dubslow: I must have crossed with your post. If you can get me easy to understand info, I'll modify AliWin2. Right now the submission is performed by Aliqueit, but I see no reason that I can't construct a wget line to submit the elf file directly to the db.

[Dubslow]

Originally Posted by EdH

Sorry, I'm still a little confused. Are you saying that logging in to the db via a browser, which will set a cookie, and then using the aliqueit -s option, will allow the data to be accepted as from a logged in user?

I'm not sure. I know that Python and Perl scripts cannot use any cookies kept by the browsers (not with the default library functions, anyways; if you knew where a browser kept its files, you might be able to hack it).

Originally Posted by EdH

Or, is there a way to send login info along with the factors via wget?

Would be nice, but I have yet to find a reference for all the API options available for FDB (something like what PrimeNet has here. Syd?)

Originally Posted by EdH

AliWin2 simply uses wget via the -s option in Aliqueit to submit lines from the elf, currently. I make no attempt to log into the db before using the -s option. And, I don't know how to tell if lines submitted were credited to my login.

Thanks.

@Dubslow: I must have crossed with your post. If you can get me easy to understand info, I'll modify AliWin2. Right now the submission is performed by Aliqueit, but I see no reason that I can't construct a wget line to submit the elf file directly to the db.

You'll have to Google wget yourself, but the example I posted seemed straightforward enough. For what it's worth, here's the aliqueit code:

//submits the elf file belonging to sequence <seq> to Syd's DB.
//only sequence iterations >= <from_iteration> are sent.
bool submit_elf( mpz_class & seq, int from_iteration ) {
	ifstream f( get_elf_name( seq ).c_str() );
	if( !f.is_open() ) {
		cout << "ERROR: couldn't open elf file: " << get_elf_name( seq ) << endl;
		return false;
	}
	string line;
	string post_data = "report=true&msub=";
	//string post_data = "report=true&background=on&msub=";
	int sent_lines = 0;
	while( getline( f, line ) ) {
		if( atoi( line.c_str() ) >= from_iteration ) {
			post_data += urlencode( line + "\n" );
			++sent_lines;
		}
	}
	f.close();

	if( !sent_lines ) {
		cout << "WARNING: no lines to send." << endl;
		return false;
	}

	string post_file = "aliqueit_tmp_post_" + seq.get_str();
	string tmp_file = "aliqueit_tmp_wget_" + seq.get_str();
	ofstream fo( post_file.c_str() );
	fo << post_data;
	fo.close();
	cout << "Sending " << sent_lines << " lines..." << endl;
	system( ( "wget --cache=off --output-document=" + tmp_file + " --post-file=" + post_file + " http://factorization.ath.cx/search.php?report=true" ).c_str() );
	delete_file( post_file.c_str() );

	f.clear();
	f.open( tmp_file.c_str() );
	while( getline( f, line ) ) {
		size_t o = line.find( "Found " );
		if( o != line.npos ) {
			cout << "DB reports " << atoi( line.substr( o + 6 ).c_str() ) << " new factors." << endl;
		}
	}
	f.close();
	delete_file( tmp_file.c_str() );
	return true;
}

(This can be found at line 875 of aliqueit.cc, version 1.12. I also don't know enough C++/functions he's using/wget (etc.) to understand exactly what he's submitting to the DB, though I can make an educated guess.)

[EdH]

At http://www.mail-archive.com/wget@sun.../msg11211.html, I found the following:

...
Otherwise, you can perform the login using Wget, saving the cookies to a
file of your choice, using --post-data=..., --save-cookies=cookies.txt,
and probably --keep-session-cookies. This will require that you know
what data to place in --post-data, which generally requires that you dig
around in the HTML to find the right form field names, and where to post
them.

For instance, if you find a form like the following within the page
containing the log-in form:

<form action="/doLogin.php" method="POST">
  <input type="text" name="s-login">
  <input type="password" name="s-pass">
</form>

then you need to do something like:

  $ wget --post-data='s-login=USERNAME&s-pass=PASSWORD' \
    --save-cookies=my-cookies.txt --keep-session-cookies \
    http://HOSTNAME/doLogin.php

(Note that you _don't_ necessarily send the information to the page that
 had the login page: you send it to the spot mentioned in the "action"
attribute of the password form.)

Once this is done, you _should_ be able to perform further operations
with Wget as if you're logged in, by using

  $ wget --load-cookies=my-cookies.txt --save-cookies=my-cookies.txt \
    --keep-session-cookies ...
...

The db's form is:

<form action="login.php" method="POST"><table border=0 width="98%"><tr><td align="center" colspan=2 bgcolor="#BBBBBB">Login</td></tr>
<tr><td bgcolor="#DDDDDD">Loginname</td> <td bgcolor="#DDDDDD"><input type="text" name="user" value="" size=20></td> </tr>
<tr><td>Password</td> <td><input type="password" name="pass" value="" size=20></td> </tr>
<tr><td align="center" colspan=2 bgcolor="#DDDDDD"><input type="submit" value="Login" name="dlogin"></td> </tr>
<tr><td height=20 colspan=3></td></tr>
<tr><td align="center" colspan=2>No account? <a href="login.php?register=1">Register here</a></td> </tr></table></form>

From the above, I would guess that the wget line to log in would be:

wget --post-data='user=USERNAME&pass=PASSWORD' \
--save-cookies=my-cookies.txt --keep-session-cookies \
http://www.factordb.com/login.php

Then, parsing the elf and constructing the rest of the wget send line should follow what Aliqueit is doing...

Hopefully, I'll have a chance to try this in the next few days...