Scratch Chess Engine - Game of Kings

cuckoopig

cuckoopig wrote:

when green flag clicked

forever
say [hi]
end

12Cashews

sup

meonguy1

I have a challenge for GoK to solve… k2rr3/1p6/pqp4p/4Q1p1/RnBp4/3P3P/2P2PP1/1R4K1 w - - 4 29 anyways what is changed in GoK 3.66?

ArnoHu

meonguy1 wrote:
I have a challenge for GoK to solve… k2rr3/1p6/pqp4p/4Q1p1/RnBp4/3P3P/2P2PP1/1R4K1 w - - 4 29 anyways what is changed in GoK 3.66?

Best move is Qf6, GoK finds it on TurboWarp DIfficult after 16 seconds on my system.

GoK 3.66:

Removed full search extension for checks (now covered by quiescence) and forks
Code cleanup quiescence standing pat

ArnoHu

GoK Medium draws against Bonsai Blue on TurboWarp

GoK does not always win, and here is a game it actually should have lost against Bonsai: https://lichess.org/hlgccsA7 (GoK Medium playing white). Several mistakes, but the decisive blunder happens at 15. Nd5. However I could only partly reproduce it later, as GoK found a better move after 5 seconds on replay, and Medium has 10 seconds think time. Maybe the JavaScript engine went into garbage collection suspension for some seconds. Still this is something to look into. At the end it was a draw.

Grijfland

ArnoHu wrote:
meonguy1 wrote:
I have a challenge for GoK to solve… k2rr3/1p6/pqp4p/4Q1p1/RnBp4/3P3P/2P2PP1/1R4K1 w - - 4 29 anyways what is changed in GoK 3.66?

Best move is Qf6, GoK finds it on TurboWarp DIfficult after 16 seconds on my system.

GoK 3.66:
Removed full search extension for checks (now covered by quiescence) and forks
Code cleanup quiescence standing pat

On my computer (and Turbowarp - Difficult 40 sec) “GoK” plays Qa5 first. But if we now put the situation back on the board (Import), Qf6 is immediately considered and played !!. So is it recommended to re-enter the position (Import) to get GoK in a higher ply?

ArnoHu

Grijfland wrote:
ArnoHu wrote:
meonguy1 wrote:
I have a challenge for GoK to solve… k2rr3/1p6/pqp4p/4Q1p1/RnBp4/3P3P/2P2PP1/1R4K1 w - - 4 29 anyways what is changed in GoK 3.66?

Best move is Qf6, GoK finds it on TurboWarp DIfficult after 16 seconds on my system.

GoK 3.66:
Removed full search extension for checks (now covered by quiescence) and forks
Code cleanup quiescence standing pat

On my computer (and Turbowarp - Difficult 40 sec) “GoK” plays Qa5 first. But if we now put the situation back on the board (Import), Qf6 is immediately considered and played !!. So is it recommended to re-enter the position (Import) to get GoK in a higher ply?

Not necessarily. But the first run fills up its transposition table, which will be reused the second time. What I often do is to important the board one move before the situation under analysis and let the engine make that preceding move, in order to have a realistic transposition table state.

Grijfland

ArnoHu wrote:
Grijfland wrote:
ArnoHu wrote:
meonguy1 wrote:
I have a challenge for GoK to solve… k2rr3/1p6/pqp4p/4Q1p1/RnBp4/3P3P/2P2PP1/1R4K1 w - - 4 29 anyways what is changed in GoK 3.66?

Best move is Qf6, GoK finds it on TurboWarp DIfficult after 16 seconds on my system.

GoK 3.66:
Removed full search extension for checks (now covered by quiescence) and forks
Code cleanup quiescence standing pat

On my computer (and Turbowarp - Difficult 40 sec) “GoK” plays Qa5 first. But if we now put the situation back on the board (Import), Qf6 is immediately considered and played !!. So is it recommended to re-enter the position (Import) to get GoK in a higher ply?

Not necessarily. But the first run fills up its transposition table, which will be reused the second time. What I often do is to important the board one move before the situation under analysis and let the engine make that preceding move, in order to have a realistic transposition table state.

Ok Arno, I already suspected this. Good to know. And the “L” and “K” command is also an interesting tool.

ArnoHu

ArnoHu wrote:
GoK Medium draws against Bonsai Blue on TurboWarp

GoK does not always win, and here is a game it actually should have lost against Bonsai: https://lichess.org/hlgccsA7 (GoK Medium playing white). Several mistakes, but the decisive blunder happens at 15. Nd5. However I could only partly reproduce it later, as GoK found a better move after 5 seconds on replay, and Medium has 10 seconds think time. Maybe the JavaScript engine went into garbage collection suspension for some seconds. Still this is something to look into. At the end it was a draw.

This indeed was a bug in quiescent transposition table evaluation caching. Fixed in version 3.67.

ArnoHu

Game of Kings Scratch Chess wins against Stockfish 14 Level 7

IT IS DONE!

I was hoping Game of Kings (Difficult, TurboWarp, competition mode) had reached a level by now to draw once in a while against Stockfish 14 level 7. Over several games GoK fought hard and sometimes came close, but to no avail. Today I fixed a bug in its caching algorithm, which I discovered during a testgame against its favorite Scratch chess opponent Bonsai Chess (for those interested into details, there was some premature transposition table update with board evaluations gained during quiescence search cutoffs). After resolving the problem, I thought it might be worth to try again. Game #1 was a loss. But in game #2 GoK did more than just drawing, it won in 93 moves! Here is the replay of the game on lichess (GoK playing black): https://lichess.org/3TiyUsL7

This is a major milestone for project which started as a very limited engine of 1,700 blocks, and now is a sophisticated machine of more than 12,000 blocks.

Big thanks go to:
@Grijfland for creating the opening book, and for pushing GoK to its limits and providing a lot of helpful feedback
@Samueldora for the great Bonsai project, and for exchanging ideas
@GarboMuffin for creating the super-fast TurboWarp Scratch runtime

Last edited by ArnoHu (Jan. 22, 2022 23:11:21)

ArnoHu

ArnoHu wrote:
Game of Kings Scratch Chess wins against Stockfish 14 Level 7

IT IS DONE!

I was hoping Game of Kings (Difficult, TurboWarp, competition mode) had reached a level by now to draw once in a while against Stockfish 14 level 7. Over several games GoK fought hard and sometimes came close, but to no avail. Today I fixed a bug in its caching algorithm, which I discovered during a testgame against its favorite Scratch chess opponent Bonsai Chess (for those interested into details, there was some premature transposition table update with board evaluations gained during quiescence search cutoffs). After resolving the problem, I thought it might be worth to try again. Game #1 was a loss. But in game #2 GoK did more than just drawing, it won in 93 moves! Here is the replay of the game on lichess (GoK playing black): https://lichess.org/3TiyUsL7

This is a major milestone for project which started as a very limited engine of 1,700 blocks, and now is a sophisticated machine of more than 12,000 blocks.

Big thanks go to:
@Grijfland for creating the opening book, and for pushing GoK to its limits and providing a lot of helpful feedback
@Samueldora for the great Bonsai project, and for exchanging ideas
@GarboMuffin for creating the super-fast TurboWarp Scratch runtime

Having thought it over, I declare the GoK Chess Engine to be finished at this point. I might improve the user interface a bit in the future, but I don't plan to touch the engine code any more (unless there is a significant bug which would lead to severely affected gameplay, which is quite unlikely).

meonguy1

imma try to tweak its pieces values and see whether it actually improve.

meonguy1

ArnoHu wrote:
Game of Kings Scratch Chess wins against Stockfish 14 Level 7

IT IS DONE!

I was hoping Game of Kings (Difficult, TurboWarp, competition mode) had reached a level by now to draw once in a while against Stockfish 14 level 7. Over several games GoK fought hard and sometimes came close, but to no avail. Today I fixed a bug in its caching algorithm, which I discovered during a testgame against its favorite Scratch chess opponent Bonsai Chess (for those interested into details, there was some premature transposition table update with board evaluations gained during quiescence search cutoffs). After resolving the problem, I thought it might be worth to try again. Game #1 was a loss. But in game #2 GoK did more than just drawing, it won in 93 moves! Here is the replay of the game on lichess (GoK playing black): https://lichess.org/3TiyUsL7

This is a major milestone for project which started as a very limited engine of 1,700 blocks, and now is a sophisticated machine of more than 12,000 blocks.

Big thanks go to:
@Grijfland for creating the opening book, and for pushing GoK to its limits and providing a lot of helpful feedback
@Samueldora for the great Bonsai project, and for exchanging ideas
@GarboMuffin for creating the super-fast TurboWarp Scratch runtime

WOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

meonguy1

anyways its kinda done

meonguy1 wrote:
imma try to tweak its pieces values and see whether it actually improve.

https://turbowarp.org/632475744
press o to show pieces values

meonguy1

ok the bug of piece rendering is somewhat fixed. now the pieces are shown if you are playing white… the problem hasnt been fixed if you are playing black though

Grijfland

Very nice Arno for your great programming work. Personally, I believe that with GoK Chess we are not yet at its highest level. There will be more about this in this theater in the near future.

ArnoHu

Grijfland wrote:
Very nice Arno for your great programming work. Personally, I believe that with GoK Chess we are not yet at its highest level. There will be more about this in this theater in the near future.

That could be, curious to hear your suggestions. The thing is, most additional improvements of late (with the exception of quiescent search checks and evasions) came at such a high performance cost on Scratch, which rendered them unfeasible.

Grijfland

This victory is a beautiful and a memorable moment, but says nothing about the definitive playing strength of GoK.
In the 80s and 90s of the last century, we learned that playing chess matches between computer programs says
nothing about their playing strength. That is why tests were devised, from which conclusions can be drawn
about the playing strength of a chess engine. For example, there is a Quicktest of 24 chess positions, (Michael Scheidt's Quicktest 2002)
where GoK can only solve 2 (the horizon effect of an engine does not even play an important role,
nor does the amount of possible moves that have to be calculated). One of such chess position is this one,

3Q4/3p4/P2p4/N2b4/8/4P3/5p1p/5Kbk w - -

which I don't really understand why GoK can't calculate the solution.
Somewhere in the engine a knob has to be turned to take Qa8 into consideration.
When you make the move with white, you see that black first values his position highly,
only to see immediately afterwards that white is suddenly much better, despite the loss of the queen on Qa8.
The diagonal a8-h1 is very important in this position. This must be blocked for black.
The course of the moves can go as follows.
Qa8 - Bxa8 / Nb7 - d5 / Nd6 - d4 / e4 - Bc6 / a7 - Ba4 / a8 (Q) - Bc6 /….. (White will now easily win).
When will GoK consider and play the move Qa8?

ArnoHu

Grijfland wrote:
This victory is a beautiful and a memorable moment, but says nothing about the definitive playing strength of GoK.
In the 80s and 90s of the last century, we learned that playing chess matches between computer programs says
nothing about their playing strength. That is why tests were devised, from which conclusions can be drawn
about the playing strength of a chess engine. For example, there is a Quicktest of 24 chess positions, (Michael Scheidt's Quicktest 2002)
where GoK can only solve 2 (the horizon effect of an engine does not even play an important role,
nor does the amount of possible moves that have to be calculated). One of such chess position is this one,

3Q4/3p4/P2p4/N2b4/8/4P3/5p1p/5Kbk w - -

which I don't really understand why GoK can't calculate the solution.
Somewhere in the engine a knob has to be turned to take Qa8 into consideration.
When you make the move with white, you see that black first values his position highly,
only to see immediately afterwards that white is suddenly much better, despite the loss of the queen on Qa8.
The diagonal a8-h1 is very important in this position. This must be blocked for black.
The course of the moves can go as follows.
Qa8 - Bxa8 / Nb7 - d5 / Nd6 - d4 / e4 - Bc6 / a7 - Ba4 / a8 (Q) - Bc6 /….. (White will now easily win).
When will GoK consider and play the move Qa8?

Correct, it says nothing about the definitive playing strength. But it proves that a Scratch Chess engine can win against Stockfish 14 level 7, and that proof is what is new, and was my goal all the way (well actually I was just aiming for a draw).

Now for special tests like the above-mentioned - that's checkmate in 9 full movies (18 plies). Checkmate verification takes another 2 plies, so overall that's a full search depth of 20 plies. GoK reaches depth 15 on TurboWarp on that board - that would even be enough, if everything after move 15 had been checks, captures and evasions, but that is not the case. Also, a queen sacrifice at the beginning for sure triggers futility pruning, which is a good thing in 99% of cases, but certainly hurts here, as in the other 1% of cases. I don't see any other way to implement coverage for this test case than by search depth. And on Scratch (well TurbowWarp) we simply cannot get to ply 20 on complex boards, involving pawn promotions, unless we throw out a lot of evaluation code, remove futility pruning and somehow magically manage to speed up move generator. Let's say we are able to do all that and pass that test. Still we are going lose many other games we would have won otherwise, because evaluation had lost quality and sophistication, and because of the lack of futility pruning.

Everything I do engine code has advantages and drawbacks. It might be beneficial in that very situation, but hurt in others. It is always a tradeoff and a compromise. in the end there is one number, one evaluation for material, position and mobility combined, and one can always be traded for the other. I actually had to revert several improvements already, as they did not turn out net positive for playing strength. For Scratch it gets even more difficult, as every block execution just adds a lot of performance overhead.

Last edited by ArnoHu (Jan. 23, 2022 14:19:54)

Grijfland

ArnoHu wrote:
Grijfland wrote:
This victory is a beautiful and a memorable moment, but says nothing about the definitive playing strength of GoK.
In the 80s and 90s of the last century, we learned that playing chess matches between computer programs says
nothing about their playing strength. That is why tests were devised, from which conclusions can be drawn
about the playing strength of a chess engine. For example, there is a Quicktest of 24 chess positions, (Michael Scheidt's Quicktest 2002)
where GoK can only solve 2 (the horizon effect of an engine does not even play an important role,
nor does the amount of possible moves that have to be calculated). One of such chess position is this one,

3Q4/3p4/P2p4/N2b4/8/4P3/5p1p/5Kbk w - -

which I don't really understand why GoK can't calculate the solution.
Somewhere in the engine a knob has to be turned to take Qa8 into consideration.
When you make the move with white, you see that black first values his position highly,
only to see immediately afterwards that white is suddenly much better, despite the loss of the queen on Qa8.
The diagonal a8-h1 is very important in this position. This must be blocked for black.
The course of the moves can go as follows.
Qa8 - Bxa8 / Nb7 - d5 / Nd6 - d4 / e4 - Bc6 / a7 - Ba4 / a8 (Q) - Bc6 /….. (White will now easily win).
When will GoK consider and play the move Qa8?

Correct, it says nothing about the definitive playing strength. But it proves that a Scratch Chess engine can win against Stockfish 14 level 7, and that proof is what is new, and was my goal all the way (well actually I was just aiming for a draw).

Now for special tests like the above-mentioned - everything I do in the engine code has advantages and drawbacks. It might be beneficial in that very situation, but hurt in others. It is always a tradeoff. in the end there is one number, one evaluation for material, position and mobility combined, and one can always be traded for the other. I actually had to revert several improvements already, as they did not turn out net positive for playing strength. For Scratch it gets even more difficult, as every block execution just adds a lot of performance voerhead.

Okay Arno, I understand this. But is there a version of GoK, in which this chess position is seen by GoK? Or try setting the engine in such a way that it can solve this problem and then let this test version play against version 3.67

Discuss Scratch