mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Hobbies > Chess

Reply
 
Thread Tools
Old 2021-02-07, 13:28   #12
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

25×33 Posts
Default

That's a neat idea, scid has an ECO file defining ~10k opening lines over half of which are over 10 ply long. Not a quick thing to implement but a quick thing to execute and potentially strong.



I've also taken a peek at stockfish. There doesn't seem to be an easy way to generate an ordered list of moves for a position but it's simple enough to give it a position and it returns the best calculated move using the UCI interface interface. With a depth of one some of us might even still be alive by the time all ~1/3 billion positions in caissabase are evaluated in this way. It would certainly accelerate the need for concurrency via OpenMP, something else on the TODO if I'm not bored of this by then.



Stockfish-bestmove and an opening book can both be integrated with the existing sortv1 ordering so they seem like the natural way to progress. If stockfish is quick enough the plan is to merge stockfish with v1 forming v2, merge ECO with v1 forming v3, and merging all three into v4 to then allow for some juicy comparisons.
M344587487 is offline   Reply With Quote
Old 2021-02-15, 13:53   #13
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

25×33 Posts
Default

An ordered list of moves for a position can be extracted from stockfish, by setting MultiPV to the number of valid moves of the position before telling SF to evaluate. If stockfish is used at all then using the full ordering is the way to go, otherwise we're throwing away a stronger evaluation for no reason. At depth 1 (where depth is a multiplier) it does require ~1TB of UCI communication to my code and ~35GB to stockfish for Caissabase, if this ~1TB is a problem stockfish can be modified to reduce this by an order of magnitude quite easily. At depth 1 it would take ~50 hours of a single Zen2 core to process Caissabase, so PGN splitting is implemented as a quick-and-dirty way of being able to parallelise, allowing an 8 core Zen2 to process Caissabase in ~8 hours.

The pgn can be converted to ila with stockfish sort but there's an as yet undiagnosed problem which means that converting from ila fails. Every sort method shares the same code path for non-sort operations, stockfish ordering appears to be deterministic, and the other sort algorithms can be encoded/decoded at will, so the bug hunting spear needs sharpening.

There's an extra way to squeeze more juice out of a generic compressor compressing ISA form, an ordering of the games increasing commonalities between adjacent games. Possibly as simple as ordering by the average index per game, next on the TODO list.
M344587487 is offline   Reply With Quote
Old 2021-09-16, 19:34   #14
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

25×33 Posts
Default

Had a bit of a noodle again with this project. After cursing at past me for creating such an awful spider's nest of code the core engine has been re-implemented to fully use u64 bitboards (before the pieces were stored as u8 positions with bitboards only used partly). The new implementation is at least 10x quicker at generating moves and hasn't been fully optimised yet so that's a win. It makes heavy use of code like this:
Code:
while(piece_type){
  index=__builtin_ctzll(piece_type);//bitscan
  //test piece
  piece_type^=(1ull<<index);//pop
}
to pop the positions of all pieces of a certain type to do tests with (the tests use precomputed masks where possible). The implementation only uses basic bit manipulation as I don't know the more advanced concepts, but there's also the possibility of parallelising part of this implementation to AVX2 which may not be as easy with a more complex representation. AVX2 is a tempting rabbit hole even if it's not the best way forwards.

If compression is ever touched on again the first port of call will probably be storing the entire set as a de-duplicated n-ary tree using dwarf's representation which seems neat ( https://eli.thegreenplace.net/2011/0...thm-from-dwarf ). Then it's probably back to trying to cheaply rank moves in a position to make them more compressible.

The nest of code still exists, it's just been added to so that future me has a reason to drink.
M344587487 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Buddy compression SELROC GPU Computing 2 2019-03-08 12:58
Engine-assisted game vs Stockfish, move 36 discussion. Drawn game? MooMoo2 Other Chess Games 10 2017-04-29 17:39
GPU optimization over snappy compression ihavethepotenti Programming 2 2013-02-15 00:20
video compression chatter by jasong jasong jasong 24 2008-01-13 11:23
Compression program Prime95 LMH > 100M 1 2005-03-23 15:18

All times are UTC. The time now is 02:14.


Thu Oct 28 02:14:58 UTC 2021 up 96 days, 20:43, 0 users, load averages: 2.36, 2.55, 2.63

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.