Discuss Scratch

ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Just for the fun of it, I am letting all working Scratch chess engines play against Leela Chess Zero, the open source sister of Google AlphaZero. Under normal conditions the Scratch engines should not stand a chance, but as you can see, sometimes little Scratch projects can surprise even neural networks…

Game #1: LCZero (white) vs. Midecah Chess (black): https://lichess.org/hXdatdQP (LCZero wins in 15 moves)
Game #2: Game of Kings Chess (Sulfurous engine, white) vs. LCZero (black): https://lichess.org/BBdJkOoq (LCZero wins in 60 moves)
Game #3: Bonsai Chess (Sulfurous engine, white) vs. LCZero (black): https://lichess.org/0jfy5yim (LCZero wins in 51 moves)
Game #4: Game of Kings Chess (Sulfurous engine, white) vs. LCZero (black): https://lichess.org/Ssyh54j3 (Draw by threefold repetition, 52 moves)
Game #5: Bonsai Chess (Sulfurous engine, white) vs. LCZero (black): https://lichess.org/7xf9RE6h (LCZero wins in 23 moves)
Game #6: Archimedes Chess vs. vs. LCZero (black): https://lichess.org/Zpg67uAW (LCZero wins in 13 moves)
Game #7: Archimedes Chess vs. vs. LCZero (black): https://lichess.org/WaPLALGR (LCZero wins in 30 moves)
Game #8: LCZero (white) vs. The Turk (black): https://lichess.org/7H2V5vqz (LCZero wins in 29 moves)
Game #9: LCZero (white) vs. The Turk (black): https://lichess.org/UmF8n7aw (LCZero wins in 26 moves)
Game #10: LCZero (white) vs. Midecah Chess (black): https://lichess.org/016T1Z9r (LCZero wins in 22 moves)

LCZero playing at mode “Normal”. Version 266 and above.

Last edited by ArnoHu (June 18, 2018 15:52:20)

ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

In game #2, GoK actually had the chance to recover to an even position at move #40, but did not see that option due to an horizon effect.

Last edited by ArnoHu (May 11, 2018 19:12:42)

ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Nice game #3 - Bonsai fighting, but finally overwhelmed by LCZero: https://lichess.org/0jfy5yim

Last edited by ArnoHu (May 25, 2018 15:25:52)

ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Huge surprise in game #4 - GoK somehow rescues a draw by threefold repetition! Game protocol: https://lichess.org/Ssyh54j3

Last edited by ArnoHu (May 14, 2018 18:46:29)

Samueldora
Scratcher
100+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

#4 Great success for GoK! I had expected to LCZero to be already way too strong for Scratch based engines.

Last edited by Samueldora (May 12, 2018 05:50:29)

CodeBit
Scratcher
73 posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

ArnoHu wrote:

Huge surprise in game #4 - GoK somehow rescues a draw by threefold repetition!

Wow!
airetho
Scratcher
2 posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

That game 2 recovery is an incredible tactic!
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

LCZero repeats its success against Bonsai in game #5: https://lichess.org/7xf9RE6h
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Game #6 with the expected result - LCZero dominates Archimedes from the very start: https://lichess.org/Zpg67uAW
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Leela Zero reigns against Archmiedes all through game #7, but also misses several faster checkmate sequences: https://lichess.org/WaPLALGR

Last edited by ArnoHu (May 27, 2018 21:17:42)

ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Leela blunders once against The Turk, but still wins conclusively in game #8: https://lichess.org/7H2V5vqz
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Game #9: Leela repeats clear succes against The Turk: https://lichess.org/UmF8n7aw
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

And finally another quick win against Midecah Chess in game #10: https://lichess.org/016T1Z9r

This concludes our little Leela Chess Zero vs. Scratch Chess Engines test run. Thanks for watching!

Scratch Chess - Game of Kings ONLINE
BoltBait
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

ArnoHu wrote:

Leela blunders once against The Turk, but still wins conclusively in game #8: https://lichess.org/7H2V5vqz

ArnoHu wrote:

Game #9: Leela repeats clear succes against The Turk: https://lichess.org/UmF8n7aw

I'm just glad The Turk lasted more than 10 moves.
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

BoltBait wrote:

ArnoHu wrote:

Leela blunders once against The Turk, but still wins conclusively in game #8: https://lichess.org/7H2V5vqz

ArnoHu wrote:

Game #9: Leela repeats clear succes against The Turk: https://lichess.org/UmF8n7aw

I'm just glad The Turk lasted more than 10 moves.

Yeah, The Turk did pretty well! I am just afraid Leela will get even more scary over time, my chess-versed friends tell me she still has endgame weaknesses. That one draw might be a one-off forever.
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

ArnoHu wrote:

In game #2, GoK actually had the chance to recover to an even position at move #40, but did not see that option due to an horizon effect.

I did some more detailed analysis on why GoK did not prevent Leela's pawn promotion at https://lichess.org/BBdJkOoq#84, as that looks pretty obvious from a human player perspective. It was an horizon effect due to the fact that GoK saw another potential pawn promotion for itself, which would force Leela to additional king moves to capture GoK's pawn. That delayed but did not prevent Leela's promotion, and slided the end effect behind the horizon. At this point GoK looked 8 moves ahead (today's version would look 10 moves ahead on really fast runtimes/machines), but that was still not sufficient.
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

I have changed Game of Kings' king safety implementation lately, and decided to let it play another test game against Leela Chess Zero neural network chess AI (ID 459). This again ended up in a draw, with Leela showing quite an endgame weakness. And I concluded that GoK really needs an “offer draw” function, or go for threefold repetition, once it is less than a pawn up after move 58. Here is the complete game: https://lichess.org/2Tc5Efx1
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

ArnoHu wrote:

I have changed Game of Kings' king safety implementation lately, and decided to let it play another test game against Leela Chess Zero neural network chess AI (ID 459). This again ended up in a draw, with Leela showing quite an endgame weakness. And I concluded that GoK really needs an “offer draw” function, or go for threefold repetition, once it is less than a pawn up after move 58. Here is the complete game: https://lichess.org/2Tc5Efx1

As usual, this was played by using the faster Sulfurous Scratch runtime, which allows GoK to look up to ten moves ahead.
ArnoHu
Scratcher
1000+ posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

With an improved king safety implementation, GoK scores its third draw against Leela Chess Zero, after being at the brink of defeat: https://lichess.org/6JKISxMN

Last edited by ArnoHu (July 4, 2018 23:21:04)

seanpsean208
Scratcher
34 posts

Leela Chess Zero Neural Network AI vs. Scratch Chess Engines

Crazy that Leela blunders more than a legitimate human

Powered by DjangoBB