birdracerthree wrote:

ArnoHu wrote:

Great 15sec game GoK NNUE (black) against Delta, GoK had a dangerous situation in defence (and a blunder shifting back and forth between two close evaluations), but prevailed:

https://lichess.org/Qy18DWtW#100

I always tell myself “HCE is done and ”NNUE is mandatory" until I see games like these. I know Delta is NNUE but still…

What do you think is the #1 reason why Element loses its game against GoK NNUE? Should I prioritize depth (NMP) or check extensions?

I think the #1 reason is still search depth, I would personally consider depth 10 (with LMR, NMP, TT, ID, etc.) to be the minimum for “complete” tactical awareness. Well, I say “complete”, but it's more like 98% tactical awareness, as in, you won't take free pieces that lead to an attack / mate, you won't “sacrifice” a piece without seeing it's refutation, etc.

But tactical awareness in it's purest form isn't why I think depth 8-10 is really good, but rather is it's positional benefit, and sometimes all it takes is just 1-2 more moves to help a positions static eval go from -100, to +200, and these don't even have to be captures! These can be a “quiet” arrangement of pieces around the opponents kings, or stacking the rooks on a open file while blocking the opponent's passer at the same time!

To be honest, I think that Element has one of the best static evals of any engine, so all it needs is a slight boost in search depth with NMP, and LMR, and then add check extensions / checks in q-search later on.




(And yes, I also sometimes think that HCE will never be useful compared to NNUE, but then WD finds the brilliant Bxh7 on move 13 at around 6 seconds. https://lichess.org/Qy18DWtW#26 But, GoK NNUE's positional play and ability to save lost positions is just simply unmatched.)

ArnoHu wrote:

Great 15sec game GoK NNUE (black) against Delta, GoK had a dangerous situation in defence (and a blunder shifting back and forth between two close evaluations), but prevailed:

https://lichess.org/Qy18DWtW#100

Nice game (looks very good in my database)

birdracerthree wrote:

I always tell myself “HCE is done and ”NNUE is mandatory" until I see games like these. I know Delta is NNUE but still…

Making an engine stronger which gets its strength out of good evaluation is a little different than classical. Blunders happen when the network does not understand something, often having reasons hce does not consider until reaching a much higher depth…
I don't get what you mean with “games like these”, in my opinion the game was well played and contained reasonable blunders.

S_P_A_R_T_Test wrote:

birdracerthree wrote:

ArnoHu wrote:

Great 15sec game GoK NNUE (black) against Delta, GoK had a dangerous situation in defence (and a blunder shifting back and forth between two close evaluations), but prevailed:

https://lichess.org/Qy18DWtW#100

I always tell myself “HCE is done and ”NNUE is mandatory" until I see games like these. I know Delta is NNUE but still…

What do you think is the #1 reason why Element loses its game against GoK NNUE? Should I prioritize depth (NMP) or check extensions?

I think the #1 reason is still search depth, I would personally consider depth 10 (with LMR, NMP, TT, ID, etc.) to be the minimum for “complete” tactical awareness. Well, I say “complete”, but it's more like 98% tactical awareness, as in, you won't take free pieces that lead to an attack / mate, you won't “sacrifice” a piece without seeing it's refutation, etc.

But tactical awareness in it's purest form isn't why I think depth 8-10 is really good, but rather is it's positional benefit, and sometimes all it takes is just 1-2 more moves to help a positions static eval go from -100, to +200, and these don't even have to be captures! These can be a “quiet” arrangement of pieces around the opponents kings, or stacking the rooks on a open file while blocking the opponent's passer at the same time!

To be honest, I think that Element has one of the best static evals of any engine, so all it needs is a slight boost in search depth with NMP, and LMR, and then add check extensions / checks in q-search later on.




(And yes, I also sometimes think that HCE will never be useful compared to NNUE, but then WD finds the brilliant Bxh7 on move 13 at around 6 seconds. https://lichess.org/Qy18DWtW#26 But, GoK NNUE's positional play and ability to save lost positions is just simply unmatched.)

I don’t know why, but I have really struggled to implement anything into Element. I tried LMR and the number of nodes went up somehow, Element’s negamax is turn based, so NMP is unnecessarily difficult to implement. I will try again and I will see how things go…

Good news : Quiescence looks like working (finally it does not move the human player's pieces)
Bad news : It's so SLOW

Least complex WD game be like: https://lichess.org/study/ao5waq0z/880F9Wjp#0

To be honest, I don't even know how to debug / fix the blunders in this game, they're the strangest moves I've seen in a while!

birdracerthree wrote:

ArnoHu wrote:

Great 15sec game GoK NNUE (black) against Delta, GoK had a dangerous situation in defence (and a blunder shifting back and forth between two close evaluations), but prevailed:

https://lichess.org/Qy18DWtW#100

I always tell myself “HCE is done and ”NNUE is mandatory" until I see games like these. I know Delta is NNUE but still…

What do you think is the #1 reason why Element loses its game against GoK NNUE? Should I prioritize depth (NMP) or check extensions?

I understand what you mean. Although here I must report, GoK's mistakes where caused by a bug in TT horizon calculation. Not in the TT code, but in the caller (so no worries if you are using GoK TT, it is just fine).

GoK NNUE (TW-Ext, Difficult) against Stockfish 16 level in lichess.org live game:
https://lichess.org/broadcast/gok-nnue-vs-stockfish/round-1/Yi0Qj1sB/QBt3NPCB

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

ArnoHu wrote:

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

Cool stuff!

Is a rematch against SF level 7 going to be played?

ArnoHu wrote:

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

Strong! I cannot wait to come back and make delta stronger, until then!

Delta 4 checkpoint??

FastSkript wrote:

ArnoHu wrote:

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

Strong! I cannot wait to come back and make delta stronger, until then!

Delta 4 checkpoint??

Can't wait

Strong game by White Dove (P2, black) against GoK NNUE (Medium), until it blunders during endgame. 96% vs. 90% accuracy:
https://lichess.org/pH64XHQd#72

ArnoHu wrote:

Strong game by White Dove (P2, black) against GoK NNUE (Medium), until it blunders during endgame. 96% vs. 90% accuracy:
https://lichess.org/pH64XHQd#72

Last game in series against Element (6+8), GoK NNUE again at 98% vs. 87%:
https://lichess.org/7SZyIES4#57

Four games in a row at 96%+ against strong opponents.

FastSkript wrote:

ArnoHu wrote:

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

Strong! I cannot wait to come back and make delta stronger, until then!

Delta 4 checkpoint??

What do you think has most improvement potential for Delta?

Those three bugs had all been lingering in GoK NNUE since it was created (two of them came from Classic). I will only do UCI-bulk-test confirmed changes from now on.

GoK Classic (Medium, white) had a huge challenge during early midgame, when inexplicably its search depth was reduced from 10 to 6 for some moves and it blundered against Delta (15s). However GoK recovered and still won, 91% vs. 89% accuracy:
https://lichess.org/lnZafXBW#147

Let's not forget Classic's good track record of late, except when playing against GoK NNUE. And it also benefits from two out of three of NNUE's bugfixes.

ArnoHu wrote:

GoK Classic (Medium, white) had a huge challenge during early midgame, when inexplicably its search depth was reduced from 10 to 6 for some moves and it blundered against Delta (15s). However GoK recovered and still won, 91% vs. 89% accuracy:
https://lichess.org/lnZafXBW#147

Let's not forget Classic's good track record of late, except when playing against GoK NNUE. And it also benefits from three out of three of NNUE's bugfixes.

When did the search depth decrease?

birdracerthree wrote:

ArnoHu wrote:

GoK Classic (Medium, white) had a huge challenge during early midgame, when inexplicably its search depth was reduced from 10 to 6 for some moves and it blundered against Delta (15s). However GoK recovered and still won, 91% vs. 89% accuracy:
https://lichess.org/lnZafXBW#147

Let's not forget Classic's good track record of late, except when playing against GoK NNUE. And it also benefits from two out of three of NNUE's bugfixes.

When did the search depth decrease?

16 - 18. Probably exactly because eval went down, and your PV search suddenly produces a large search window for non-PV nodes.

S_P_A_R_T_Test wrote:

ArnoHu wrote:

ArnoHu wrote:

GoK NNUE (Medium, white) with a perfect game against Delta (15sec), at 98% vs. 91% accuracy:
https://lichess.org/Xl88x4tb#79

A perfect game against Delta is no easy feat.

Rematch, GoK black, another perfect game - 99% for GoK, 87% for Delta:
https://lichess.org/zAValfBK#58

The unfortunate match against Stockfish aside, the bugfixes for

• Incorrect NN InputWeight index mapping for one piece-type
  
• Pseudo-legal move handling for hash moves
  
• Transposition table horizon calculation
  

are indeed showing effect.

For everyone interested, here are the three commits:

https://github.com/ArnoHue/scratch/commit/2ceac91f8ca9831b1c8da31e0e561cfd24f1be22
https://github.com/ArnoHue/scratch/commit/673484005db70d56e70cc4f0cabf3f65233d9f32
https://github.com/ArnoHue/scratch/commit/deb141f2e9846662583eb0983c9ef2bcc2941556

Cool stuff!

Is a rematch against SF level 7 going to be played?

Yes, certainly. Although in yesterday's game, GoK already had all the bugfixes.