mersenneforum.org Trial Factoring for Dummies
 Register FAQ Search Today's Posts Mark Forums Read

2020-05-08, 19:58   #67
chalsall
If I May

"Chris Halsall"
Sep 2002

928610 Posts

Quote:
 Originally Posted by greg If TF and/or P-1 became verifiable (even if only a subset of clients) then there is the potential to reward that work with a portion of the prime prize award (even if only a \$2.56 check).

In the official notice, the TF'ers and P-1'ers (etc.) are covered in the "et al" language. But it would be kinda cool getting an official document from GIMPS for being in direct assistance for the next MP found. Just don't fall into Knuth's situation with regards to cheques...

Also, a "bump" on this discussion. Do we have any code to the stage where it could be alpha tested on kit yet? We do have some places to test this...

2020-05-08, 20:34   #68
SethTro

"Seth"
Apr 2019

181 Posts

Quote:
 Originally Posted by chalsall However, I personally feel that /some/ would value having some sort of sanity check on the GPU hardware. As said before, some of us these things for more than just TF'ing (where determinism (or, at least, sanity) is coveted).
I agree with the first sentence.
I think the 2nd sentence is missing an "are" (between "things for more") and I agree doubling with that, I've been thinking about if this could be adopted to P-1 or ECM so that a factordb like service wouldn't have to trust blindly submissions.

2020-05-08, 21:00   #69
SethTro

"Seth"
Apr 2019

181 Posts

Quote:
 Originally Posted by chalsall LOL. Not a bad idea! In the official notice, the TF'ers and P-1'ers (etc.) are covered in the "et al" language. But it would be kinda cool getting an official document from GIMPS for being in direct assistance for the next MP found. Just don't fall into Knuth's situation with regards to cheques... Also, a "bump" on this discussion. Do we have any code to the stage where it could be alpha tested on kit yet? We do have some places to test this...
Absolutely; The code is at https://github.com/sethtroisi/mfaktc/tree/proof and you can build it in the standard way.

If you use linux and cuda 10.1 I can provide a binary if you can't compile it yourself.

Everything should look like normal except the version is incremented and you should see proof lines in your results.txt

Code:
M66023333 proof_k(20475767038603731855001): 28 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(33270468931786126541321): 27 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(33150685168141475305463): 25 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(28851305727414925240879): 24 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(32228068860752704443631): 22 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
M66023333 proof_k(23676960155764888060537): 20 bits [TF:74:75:mfaktc 0.21 barrett76_mul32_gs]
[Fri May  1 22:46:32 2020]
no factor for M66023333 from 2^74 to 2^75 [mfaktc 0.21 barrett76_mul32_gs]
It would actually be nice to build a worktodo.txt that uses many different kernals and verifies they all produce proof logs.

I'll provide a program to verify the proof logs this weekend; currently it's a combination of the check in #47 for small kernels (e.g. 75bit_mul32_gs) and https://gist.github.com/sethtroisi/a2942e29be6ab43a6eeb4b4987ee3246 for larger kernels

2020-05-08, 21:19   #70
chalsall
If I May

"Chris Halsall"
Sep 2002

2·4,643 Posts

Quote:
 Originally Posted by SethTro Absolutely; The code is at https://github.com/sethtroisi/mfaktc/tree/proof and you can build it in the standard way.
Cool. I'll take a look at it (and run a few tests) this weekend.

Quote:
 Originally Posted by SethTro I'll provide a program to verify the proof logs this weekend; currently it's a combination of the check in #47 for small kernels (e.g. 75bit_mul32_gs) and https://gist.github.com/sethtroisi/a2942e29be6ab43a6eeb4b4987ee3246 for larger kernels
OK. It would be good to run this across a spectrum of kit and kernel sizes. Do you happen to have a list of candidates which should be run?

And, could you define your datastream a bit more? Should the servers keep the values for each bit level? As in, it's effectively a three-dimensional sparse matrix (Exponent, Bit-level, PoW)?

Separately, does anyone who understands both the maths and the Python understand what's being done here? Black-box to me (and I'm OK with that).

2020-05-08, 22:04   #71
SethTro

"Seth"
Apr 2019

101101012 Posts

Quote:
 Originally Posted by chalsall Cool. I'll take a look at it (and run a few tests) this weekend. OK. It would be good to run this across a spectrum of kit and kernel sizes. Do you happen to have a list of candidates which should be run? And, could you define your datastream a bit more? Should the servers keep the values for each bit level? As in, it's effectively a three-dimensional sparse matrix (Exponent, Bit-level, PoW)? Separately, does anyone who understands both the maths and the Python understand what's being done here? Black-box to me (and I'm OK with that).
I plan to do a test run over a large range of exponent at TF=65,70,75 this weekend.

Right now the code outputs multiple proofs per <exponent,bit-level>, I think the server should keep <user, exp, bit-level, min-pow> where min-pow is the "best" proof of work where best = hardest = rarest = smallest res or most zeros. I need to change the code a little bit before this happens.

I cleaned up the verify script and have a new version you can pass your results.txt file to.
https://gist.github.com/sethtroisi/4...7007d55cdca7e5

my TODO is
1. Rename proof_k to proof_test or proof (it's not easy to manipulate large ints in mkfact)
2. Change output format from "X bits" to "X difficulty"
3. Run over large range of inputs and verify proof
4. Update verify script one more time.

2020-05-09, 12:20   #72
SethTro

"Seth"
Apr 2019

181 Posts

Quote:
 Originally Posted by SethTro TODO is 1. Rename proof_k to proof_test or proof (it's not easy to manipulate large ints in mkfact) 2. Change output format from "X bits" to "X difficulty" 3. Run over large range of inputs and verify proof 4. Update verify script one more time.
These are all done now.

I ran ~50 exponents (5M to 250M) over several bitranges (58-60, 60-62, 65-67, 70-71) and verified ~458 proof statements (currently more than one can be generated per workitem).

Newest verification script is at https://gist.github.com/sethtroisi/4...7007d55cdca7e5
Attached Files
 worktodo.txt (4.3 KB, 47 views) results.txt (62.8 KB, 48 views)

2020-05-11, 18:56   #73
chalsall
If I May

"Chris Halsall"
Sep 2002

2×4,643 Posts

Quote:
 Originally Posted by SethTro These are all done now.
OK... Sweet. I love data!

However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct.

The actual proof lines are great to RegEx through -- easy to parse. Thanks for that.

It's going to be interesting running this on a few different platforms, and ensure we get full convergence on the proofs.

Anyone out there with flaky kit willing to give this a try? It would be good to get such test-runs in the dataset.

P.S. Is now the time to discuss optionally producing XML output?

2020-05-11, 22:41   #74
chalsall
If I May

"Chris Halsall"
Sep 2002

2×4,643 Posts

Quote:
 Originally Posted by chalsall However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct.
OK... I finally had some cycles to actually run your code (on a slow little 1050 also doing display work). An observation/question...

I see during the run that the different "proof" lines are generated during the run. However, the last proof isn't printed (perhaps it's not supposed to be). As an example (copied from the console where the job was run):
Code:
got assignment: exp=5500021 bit_min=65 bit_max=67 (4.08 GHz-days)
Starting trial factoring M5500021 from 2^65 to 2^66 (1.36 GHz-days)
k_min =  3353940660840
k_max =  6707881323983
Using GPU kernel "barrett76_mul32_gs"
Date    Time | class   Pct |   time     ETA | GHz-d/day    Sieve     Wait
May 11 18:14 |  260   5.7% |  0.408   6m09s |    299.71    82485    n.a.%
M5500021 proof(47281205372229406961): 33 difficulty
May 11 18:15 |  548  12.1% |  0.401   5m38s |    304.94    82485    n.a.%
M5500021 proof(41186001143328300737): 34 difficulty
May 11 18:15 |  923  20.2% |  0.406   5m11s |    301.18    82485    n.a.%
M5500021 proof(55435150410008224727): 37 difficulty
May 11 18:21 | 4619 100.0% |  0.409   0m00s |    298.97    82485    n.a.%
no factor for M5500021 from 2^65 to 2^66 [mfaktc 0.21 barrett76_mul32_gs]
tf(): total time spent:  6m 32.519s
I chose this example because the Proof of Work is only generated for ~20% of the run.

Is this by design? Or is there a way to get a useful hash for the last part of the run?

2020-05-13, 23:00   #75
chalsall
If I May

"Chris Halsall"
Sep 2002

221068 Posts

Quote:
 Originally Posted by chalsall Is this by design? Or is there a way to get a useful hash for the last part of the run?

I'd like to start doing some wide-spread test runs using this, and collecting data. But I'd like the codebase to be stable (and, of course, sane) before I do.

Thanks!

2020-05-14, 00:37   #76
SethTro

"Seth"
Apr 2019

181 Posts

Sorry I missed these messages, I strongly rely on MF emails (I get a kick out of "You will probably never see this message because it will end up in your spam folder.") and seem to have missed theses

Quote:
 Originally Posted by chalsall OK... Sweet. I love data! However, one comment (read: change request): Your mfaktc delta should print the proof lines after the datestamp line. As it is currently, parsing the data from a log file it is a pain to get the timestamp correct. The actual proof lines are great to RegEx through -- easy to parse. Thanks for that.
This is a work around for not having handled resume yet.
This weekend I'll change it so at default verbosity the best proof is printed directly after printing the statusline

Quote:
 Originally Posted by chalsall OK... I finally had some cycles to actually run your code (on a slow little 1050 also doing display work). An observation/question... I see during the run that the different "proof" lines are generated during the run. However, the last proof isn't printed (perhaps it's not supposed to be). As an example (copied from the console where the job was run): Code: got assignment: exp=5500021 bit_min=65 bit_max=67 (4.08 GHz-days) Starting trial factoring M5500021 from 2^65 to 2^66 (1.36 GHz-days) k_min = 3353940660840 k_max = 6707881323983 Using GPU kernel "barrett76_mul32_gs" Date Time | class Pct | time ETA | GHz-d/day Sieve Wait May 11 18:14 | 260 5.7% | 0.408 6m09s | 299.71 82485 n.a.% M5500021 proof(47281205372229406961): 33 difficulty May 11 18:15 | 548 12.1% | 0.401 5m38s | 304.94 82485 n.a.% M5500021 proof(41186001143328300737): 34 difficulty May 11 18:15 | 923 20.2% | 0.406 5m11s | 301.18 82485 n.a.% M5500021 proof(55435150410008224727): 37 difficulty May 11 18:21 | 4619 100.0% | 0.409 0m00s | 298.97 82485 n.a.% no factor for M5500021 from 2^65 to 2^66 [mfaktc 0.21 barrett76_mul32_gs] tf(): total time spent: 6m 32.519s I chose this example because the Proof of Work is only generated for ~20% of the run. Is this by design? Or is there a way to get a useful hash for the last part of the run?
This is by design, it prints the best proof as it discovers them, half of the time this is in the first 50% of the run, 1 in 5 times this is before 20%.

2020-05-14, 17:52   #77
chalsall
If I May

"Chris Halsall"
Sep 2002

100100010001102 Posts

Quote:
 Originally Posted by SethTro This is a work around for not having handled resume yet. This weekend I'll change it so at default verbosity the best proof is printed directly after printing the statusline
OK, great. I'll look out for that.

So you know, I've "forked" your mfaktc delta on GitHub, so I'll pull your changes once they've been applied. I want to make some changes myself, for use in Colab and BOINC runs. I'll send a "Pull request" once I'm happy with my changes.

Quote:
 Originally Posted by SethTro This is by design, it prints the best proof as it discovers them, half of the time this is in the first 50% of the run, 1 in 5 times this is before 20%.
OK, thanks for letting me know.

Since I'm more concerned about kit sanity than cheating detection, this is fine. Someone would /really/ have to be motivated to get around this, even in the cases of 20% coverage.

 Similar Threads Thread Thread Starter Forum Replies Last Post garo GPU Computing 100 2019-04-22 10:58 NBtarheel_33 GPU Computing 10 2011-10-13 00:04 odin Software 4 2010-08-08 20:23 S485122 PrimeNet 1 2007-09-06 00:52 jocelynl Math 8 2006-02-01 14:12

All times are UTC. The time now is 04:47.

Thu Oct 22 04:47:46 UTC 2020 up 42 days, 1:58, 0 users, load averages: 1.84, 1.44, 1.38