Getting reliable LL from unreliable hardware - Page 18

Prime95 · 2017-09-01, 02:44

Quote:

Originally Posted by fivemack

Yes, the original tests (eg M4423) were conducted on IBM hardware with 36-bit words (why 36-bit? because they wanted to support ten-digit decimal arithmetic as well as hex, so you need a multiple of 4 greater than 33.2ish)

I have not been following this carefully. I don't think we need to over-abbreviate. I also don't like combining multiple concepts into one field.

So, IMO P3 s.b. PRP-3 or Gerbicz-PRP-3. Residue should not contain "-dd". Add a new field called residue-checksum. Split program and program version into two fields.

Also, I'd add more useful data: Like round-off errors detected, Gerbicz errors, etc.

preda · 2017-09-01, 03:20

Quote:

Originally Posted by Prime95

I have not been following this carefully. I don't think we need to over-abbreviate. I also don't like combining multiple concepts into one field.

So, IMO P3 s.b. PRP-3 or Gerbicz-PRP-3. Residue should not contain "-dd". Add a new field called residue-checksum. Split program and program version into two fields.

Also, I'd add more useful data: Like round-off errors detected, Gerbicz errors, etc.

OK, we're advancing. How does this sound:

{ "exponent":25000000, "worktype":"PPR-3", "status":"C", "res64":"fffffffffffffffd", "residue-checksum":"c9caf7dd", "program":"gpuowl", "program-version":"1.0", "time":"Fri Sep 1 03:18:19 2017", "user":"meme", "cpu":"vega", "aid":"FF00AA00FF00AA00FF00AA00FF00AA00" }

preda · 2017-09-01, 03:27

Quote:

Originally Posted by Prime95

Also, I'd add more useful data: Like round-off errors detected, Gerbicz errors, etc.

My implem does not have "round-off errors" anymore.

Why do you think that the number of "rollbacks" is important? should that info be optional? (or maybe a bool, "any errors").

retina · 2017-09-01, 03:35

Quote:

Originally Posted by preda

... "time":"Fri Sep 1 03:18:19 2017" ...

I'm not really involved in this, so sorry for butting in, but that time format is horrible. At least make it consistent with regard to the significance of the values, start with the year and work you way down to the seconds. Also don't use text, different languages use different names for the month and day, use numbers. And the day of the week is not necessary. Maybe like this: "2017-09-01 03:18:19". And the time should be UTC based, for consistency, so no need to include the timezone identifier. [/date rant]

Prime95 · 2017-09-01, 03:39

Quote:

Originally Posted by preda

My implem does not have "round-off errors" anymore.

Why do you think that the number of "rollbacks" is important? should that info be optional? (or maybe a bool, "any errors").

Round-off-errors should be optional. I can imagine other optional data such failed Jacobi tests for LL. Shift count. If an optional field is left out either the program does not implement the feature or the value is zero.

Number of Gerbicz rollbacks indicate hardware reliability. If the same GPU ran LL tests or TF or P-1 it might prove useful to madpoo and his data-mining project.

P.S. Even the residue field is optional - in case of a PRP/Prime result.

preda · 2017-09-01, 04:26

Quote:

Originally Posted by retina

I'm not really involved in this, so sorry for butting in, but that time format is horrible. At least make it consistent with regard to the significance of the values, start with the year and work you way down to the seconds. Also don't use text, different languages use different names for the month and day, use numbers. And the day of the week is not necessary. Maybe like this: "2017-09-01 03:18:19". And the time should be UTC based, for consistency, so no need to include the timezone identifier. [/date rant]

Yes I agree. That's the output format of asctime()/ctime(), and that's what's in use right now.
What you propose is much more regular.

I do think it should be in UTC, yes.

preda · 2017-09-01, 06:37

ISO 8601: https://xkcd.com/1179/

preda · 2017-09-01, 06:58

{"exponent":25000000, "worktype":"PPR-3", "status":"C", "res64":"fffffffffffffffd", "residue-checksum":"c9caf7dd", "program":"gpuowl", "program-version":"1.0", "time":"2017-09-01 06:56:08", "errors":{"rollback":1}, "user":"meme", "cpu":"vega", "aid":"FF00AA00FF00AA00FF00AA00FF00AA00"}

changed date format, added errors as a json sub-object to allow various kinds of errors.
The result line is now over 256chars.

preda · 2017-09-01, 07:08

"program" could be a sub-object too, like:
"program":{"name":"gpuowl", "version":"1.0"}

Madpoo · 2017-09-02, 02:37

Thoughts after skimming through posts (been a busy week so I haven't been keeping up)...

For a time format, these are user generated timestamps so we should agree that it'll either send them as UTC all the time, or send using the timezone offset format. Bulkier, but imagine a situation where two people report the same "is prime!" (or "is probably prime!") very close to each other.

The server will know what time it was reported, but it could be helpful to also see the independently reported timestamp that the *client* finished it's run. Or maybe it wouldn't be helpful and nobody would care... right now the server doesn't save the timestamp in manual results, and auto results don't send it at all, but I think it'd be a "nice to have" option. Ultimately, if the server doesn't store the client's own timestamp then it might as well not send it at all.

As far as the server side goes, this is probably more for George & James but I think something pretty similar to the LL results table, but PRP results, would work fine.

The manual results parser should have no problem seeing the text coming in and parsing JSON (PHP does have a JSON deserializer, doesn't it? I was thinking it did but I haven't checked), and it'll know where to store the result based on whether it's PRP or LL. Well, for now any JSON would be from gpuOwl but hopefully such a well-structured input would catch on.

Creating a new table, I could do that... James has been the go-to guy for updating the results parser, I think.

Once the server is recording that data, then there's also some front-end work where it would show the PRP results... both on the exponent report along with the other data, and also a specific PRP report page, like the LL report page, I guess.

Prime95 · 2017-09-02, 03:16

Quote:

Originally Posted by Madpoo

As far as the server side goes, this is probably more for George & James but I think something pretty similar to the LL results table, but PRP results, would work fine.

Alternatively adding a new column to the LL_results table would work. The column would be something like "primality_test_used" and would be NULL for LL tests, PRP-3 for base-3 PRP tests.

This way the "exponent status" page and "LL results" web pages would work (although they would show the residue as coming from an LL test).

2017-09-01, 06:58	#195
preda "Mihai Preda" Apr 2015 3×457 Posts	{"exponent":25000000, "worktype":"PPR-3", "status":"C", "res64":"fffffffffffffffd", "residue-checksum":"c9caf7dd", "program":"gpuowl", "program-version":"1.0", "time":"2017-09-01 06:56:08", "errors":{"rollback":1}, "user":"meme", "cpu":"vega", "aid":"FF00AA00FF00AA00FF00AA00FF00AA00"} changed date format, added errors as a json sub-object to allow various kinds of errors. The result line is now over 256chars. Last fiddled with by preda on 2017-09-01 at 06:58

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Stockfish / Lutefisk game, move 14 poll. Hungry for fish and black pieces.	MooMoo2	Other Chess Games	0	2016-11-26 06:52
Redoing factoring work done by unreliable machines	tha	Lone Mersenne Hunters	23	2016-11-02 08:51
Unreliable AMD Phenom 9850	xilman	Hardware	4	2014-08-02 18:08
[new fish check in] heloo	mwxdbcr	Lounge	0	2009-01-14 04:55
The Happy Fish thread	xilman	Hobbies	24	2006-08-22 11:44

2017-09-01, 06:37	#194
preda "Mihai Preda" Apr 2015 10101011011₂ Posts	ISO 8601: https://xkcd.com/1179/

2017-09-01, 07:08	#196
preda "Mihai Preda" Apr 2015 3·457 Posts	"program" could be a sub-object too, like: "program":{"name":"gpuowl", "version":"1.0"}

2017-09-02, 02:37	#197
Madpoo Serpentine Vermin Jar Jul 2014 CF1₁₆ Posts	Thoughts after skimming through posts (been a busy week so I haven't been keeping up)... For a time format, these are user generated timestamps so we should agree that it'll either send them as UTC all the time, or send using the timezone offset format. Bulkier, but imagine a situation where two people report the same "is prime!" (or "is probably prime!") very close to each other. The server will know what time it was reported, but it could be helpful to also see the independently reported timestamp that the client finished it's run. Or maybe it wouldn't be helpful and nobody would care... right now the server doesn't save the timestamp in manual results, and auto results don't send it at all, but I think it'd be a "nice to have" option. Ultimately, if the server doesn't store the client's own timestamp then it might as well not send it at all. As far as the server side goes, this is probably more for George & James but I think something pretty similar to the LL results table, but PRP results, would work fine. The manual results parser should have no problem seeing the text coming in and parsing JSON (PHP does have a JSON deserializer, doesn't it? I was thinking it did but I haven't checked), and it'll know where to store the result based on whether it's PRP or LL. Well, for now any JSON would be from gpuOwl but hopefully such a well-structured input would catch on. Creating a new table, I could do that... James has been the go-to guy for updating the results parser, I think. Once the server is recording that data, then there's also some front-end work where it would show the PRP results... both on the exponent report along with the other data, and also a specific PRP report page, like the LL report page, I guess.