mersenneforum.org  

Go Back   mersenneforum.org > New To GIMPS? Start Here! > Information & Answers

Reply
 
Thread Tools
Old 2020-05-08, 19:58   #67
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

222378 Posts
Default

Quote:
Originally Posted by greg View Post
If TF and/or P-1 became verifiable (even if only a subset of clients) then there is the potential to reward that work with a portion of the prime prize award (even if only a $2.56 check).
LOL. Not a bad idea!

In the official notice, the TF'ers and P-1'ers (etc.) are covered in the "et al" language. But it would be kinda cool getting an official document from GIMPS for being in direct assistance for the next MP found. Just don't fall into Knuth's situation with regards to cheques...

Also, a "bump" on this discussion. Do we have any code to the stage where it could be alpha tested on kit yet? We do have some places to test this...
chalsall is offline   Reply With Quote
Old 2020-05-08, 20:34   #68
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

32×23 Posts
Default

Quote:
Originally Posted by chalsall View Post
However, I personally feel that /some/ would value having some sort of sanity check on the GPU hardware. As said before, some of us these things for more than just TF'ing (where determinism (or, at least, sanity) is coveted).
I agree with the first sentence.
I think the 2nd sentence is missing an "are" (between "things for more") and I agree doubling with that, I've been thinking about if this could be adopted to P-1 or ECM so that a factordb like service wouldn't have to trust blindly submissions.
SethTro is online now   Reply With Quote
Old 2020-05-08, 21:00   #69
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

32×23 Posts
Default

Quote:
Originally Posted by chalsall View Post
LOL. Not a bad idea!

In the official notice, the TF'ers and P-1'ers (etc.) are covered in the "et al" language. But it would be kinda cool getting an official document from GIMPS for being in direct assistance for the next MP found. Just don't fall into Knuth's situation with regards to cheques...

Also, a "bump" on this discussion. Do we have any code to the stage where it could be alpha tested on kit yet? We do have some places to test this...
Absolutely; The code is at https://github.com/sethtroisi/mfaktc/tree/proof and you can build it in the standard way.

If you use linux and cuda 10.1 I can provide a binary if you can't compile it yourself.

Everything should look like normal except the version is incremented and you should see proof lines in your results.txt

Code:
M66023333 proof_k(20475767038603731855001): 28 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(33270468931786126541321): 27 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(33150685168141475305463): 25 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(28851305727414925240879): 24 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(32228068860752704443631): 22 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(23676960155764888060537): 20 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
[Fri May  1 22:46:32 2020]
no factor for M66023333 from 2^74 to 2^75 [mfaktc 0.21 barrett76_mul32_gs]
It would actually be nice to build a worktodo.txt that uses many different kernals and verifies they all produce proof logs.

I'll provide a program to verify the proof logs this weekend; currently it's a combination of the check in #47 for small kernels (e.g. 75bit_mul32_gs) and https://gist.github.com/sethtroisi/a2942e29be6ab43a6eeb4b4987ee3246 for larger kernels
SethTro is online now   Reply With Quote
Old 2020-05-08, 21:19   #70
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3×55 Posts
Default

Quote:
Originally Posted by SethTro View Post
Absolutely; The code is at https://github.com/sethtroisi/mfaktc/tree/proof and you can build it in the standard way.
Cool. I'll take a look at it (and run a few tests) this weekend.

Quote:
Originally Posted by SethTro View Post
I'll provide a program to verify the proof logs this weekend; currently it's a combination of the check in #47 for small kernels (e.g. 75bit_mul32_gs) and https://gist.github.com/sethtroisi/a2942e29be6ab43a6eeb4b4987ee3246 for larger kernels
OK. It would be good to run this across a spectrum of kit and kernel sizes. Do you happen to have a list of candidates which should be run?

And, could you define your datastream a bit more? Should the servers keep the values for each bit level? As in, it's effectively a three-dimensional sparse matrix (Exponent, Bit-level, PoW)?

Separately, does anyone who understands both the maths and the Python understand what's being done here? Black-box to me (and I'm OK with that).
chalsall is offline   Reply With Quote
Old 2020-05-08, 22:04   #71
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

32×23 Posts
Default

Quote:
Originally Posted by chalsall View Post
Cool. I'll take a look at it (and run a few tests) this weekend.

OK. It would be good to run this across a spectrum of kit and kernel sizes. Do you happen to have a list of candidates which should be run?

And, could you define your datastream a bit more? Should the servers keep the values for each bit level? As in, it's effectively a three-dimensional sparse matrix (Exponent, Bit-level, PoW)?

Separately, does anyone who understands both the maths and the Python understand what's being done here? Black-box to me (and I'm OK with that).
I plan to do a test run over a large range of exponent at TF=65,70,75 this weekend.

Right now the code outputs multiple proofs per <exponent,bit-level>, I think the server should keep <user, exp, bit-level, min-pow> where min-pow is the "best" proof of work where best = hardest = rarest = smallest res or most zeros. I need to change the code a little bit before this happens.

I cleaned up the verify script and have a new version you can pass your results.txt file to.
https://gist.github.com/sethtroisi/4...7007d55cdca7e5

my TODO is
1. Rename proof_k to proof_test or proof (it's not easy to manipulate large ints in mkfact)
2. Change output format from "X bits" to "X difficulty"
3. Run over large range of inputs and verify proof
4. Update verify script one more time.
SethTro is online now   Reply With Quote
Old 2020-05-09, 12:20   #72
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

32·23 Posts
Default

Quote:
Originally Posted by SethTro View Post
TODO is
1. Rename proof_k to proof_test or proof (it's not easy to manipulate large ints in mkfact)
2. Change output format from "X bits" to "X difficulty"
3. Run over large range of inputs and verify proof
4. Update verify script one more time.
These are all done now.

I ran ~50 exponents (5M to 250M) over several bitranges (58-60, 60-62, 65-67, 70-71) and verified ~458 proof statements (currently more than one can be generated per workitem).

Newest verification script is at https://gist.github.com/sethtroisi/4...7007d55cdca7e5
Attached Files
File Type: txt worktodo.txt (4.3 KB, 86 views)
File Type: txt results.txt (62.8 KB, 91 views)
SethTro is online now   Reply With Quote
Old 2020-05-11, 18:56   #73
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3×55 Posts
Default

Quote:
Originally Posted by SethTro View Post
These are all done now.
OK... Sweet. I love data!

However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct.

The actual proof lines are great to RegEx through -- easy to parse. Thanks for that.

It's going to be interesting running this on a few different platforms, and ensure we get full convergence on the proofs.

Anyone out there with flaky kit willing to give this a try? It would be good to get such test-runs in the dataset.

P.S. Is now the time to discuss optionally producing XML output?
chalsall is offline   Reply With Quote
Old 2020-05-11, 22:41   #74
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3×55 Posts
Default

Quote:
Originally Posted by chalsall View Post
However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct.
OK... I finally had some cycles to actually run your code (on a slow little 1050 also doing display work). An observation/question...

I see during the run that the different "proof" lines are generated during the run. However, the last proof isn't printed (perhaps it's not supposed to be). As an example (copied from the console where the job was run):
Code:
got assignment: exp=5500021 bit_min=65 bit_max=67 (4.08 GHz-days)
Starting trial factoring M5500021 from 2^65 to 2^66 (1.36 GHz-days)
 k_min =  3353940660840
 k_max =  6707881323983
Using GPU kernel "barrett76_mul32_gs"
Date    Time | class   Pct |   time     ETA | GHz-d/day    Sieve     Wait
May 11 18:14 |  260   5.7% |  0.408   6m09s |    299.71    82485    n.a.%
M5500021 proof(47281205372229406961): 33 difficulty 
May 11 18:15 |  548  12.1% |  0.401   5m38s |    304.94    82485    n.a.%
M5500021 proof(41186001143328300737): 34 difficulty 
May 11 18:15 |  923  20.2% |  0.406   5m11s |    301.18    82485    n.a.%
M5500021 proof(55435150410008224727): 37 difficulty 
May 11 18:21 | 4619 100.0% |  0.409   0m00s |    298.97    82485    n.a.%
no factor for M5500021 from 2^65 to 2^66 [mfaktc 0.21 barrett76_mul32_gs]
tf(): total time spent:  6m 32.519s
I chose this example because the Proof of Work is only generated for ~20% of the run.

Is this by design? Or is there a way to get a useful hash for the last part of the run?
chalsall is offline   Reply With Quote
Old 2020-05-13, 23:00   #75
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3×55 Posts
Default

Quote:
Originally Posted by chalsall View Post
Is this by design? Or is there a way to get a useful hash for the last part of the run?
Seth, are you out there? Can you answer this question, please?

I'd like to start doing some wide-spread test runs using this, and collecting data. But I'd like the codebase to be stable (and, of course, sane) before I do.

Thanks!
chalsall is offline   Reply With Quote
Old 2020-05-14, 00:37   #76
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

32·23 Posts
Default

Sorry I missed these messages, I strongly rely on MF emails (I get a kick out of "You will probably never see this message because it will end up in your spam folder.") and seem to have missed theses

Quote:
Originally Posted by chalsall View Post
OK... Sweet. I love data!

However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct.

The actual proof lines are great to RegEx through -- easy to parse. Thanks for that.
This is a work around for not having handled resume yet.
This weekend I'll change it so at default verbosity the best proof is printed directly after printing the statusline

Quote:
Originally Posted by chalsall View Post
OK... I finally had some cycles to actually run your code (on a slow little 1050 also doing display work). An observation/question...

I see during the run that the different "proof" lines are generated during the run. However, the last proof isn't printed (perhaps it's not supposed to be). As an example (copied from the console where the job was run):
Code:
got assignment: exp=5500021 bit_min=65 bit_max=67 (4.08 GHz-days)
Starting trial factoring M5500021 from 2^65 to 2^66 (1.36 GHz-days)
 k_min =  3353940660840
 k_max =  6707881323983
Using GPU kernel "barrett76_mul32_gs"
Date    Time | class   Pct |   time     ETA | GHz-d/day    Sieve     Wait
May 11 18:14 |  260   5.7% |  0.408   6m09s |    299.71    82485    n.a.%
M5500021 proof(47281205372229406961): 33 difficulty 
May 11 18:15 |  548  12.1% |  0.401   5m38s |    304.94    82485    n.a.%
M5500021 proof(41186001143328300737): 34 difficulty 
May 11 18:15 |  923  20.2% |  0.406   5m11s |    301.18    82485    n.a.%
M5500021 proof(55435150410008224727): 37 difficulty 
May 11 18:21 | 4619 100.0% |  0.409   0m00s |    298.97    82485    n.a.%
no factor for M5500021 from 2^65 to 2^66 [mfaktc 0.21 barrett76_mul32_gs]
tf(): total time spent:  6m 32.519s
I chose this example because the Proof of Work is only generated for ~20% of the run.

Is this by design? Or is there a way to get a useful hash for the last part of the run?
This is by design, it prints the best proof as it discovers them, half of the time this is in the first 50% of the run, 1 in 5 times this is before 20%.
SethTro is online now   Reply With Quote
Old 2020-05-14, 17:52   #77
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

222378 Posts
Default

Quote:
Originally Posted by SethTro View Post
This is a work around for not having handled resume yet. This weekend I'll change it so at default verbosity the best proof is printed directly after printing the statusline
OK, great. I'll look out for that.

So you know, I've "forked" your mfaktc delta on GitHub, so I'll pull your changes once they've been applied. I want to make some changes myself, for use in Colab and BOINC runs. I'll send a "Pull request" once I'm happy with my changes.

Quote:
Originally Posted by SethTro View Post
This is by design, it prints the best proof as it discovers them, half of the time this is in the first 50% of the run, 1 in 5 times this is before 20%.
OK, thanks for letting me know.

Since I'm more concerned about kit sanity than cheating detection, this is fine. Someone would /really/ have to be motivated to get around this, even in the cases of 20% coverage.
chalsall is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
GPU Trial Factoring FAQ garo GPU Computing 100 2019-04-22 10:58
mfaktc for dummies NBtarheel_33 GPU Computing 10 2011-10-13 00:04
How much Trial Factoring to do? odin Software 4 2010-08-08 20:23
How far to do trial factoring S485122 PrimeNet 1 2007-09-06 00:52
trial factoring and P-1 jocelynl Math 8 2006-02-01 14:12

All times are UTC. The time now is 12:39.

Sat Dec 5 12:39:33 UTC 2020 up 2 days, 8:50, 0 users, load averages: 1.87, 1.60, 1.52

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.