Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve quality of code generated #568

Merged
merged 8 commits into from
Nov 27, 2023
Merged

Improve quality of code generated #568

merged 8 commits into from
Nov 27, 2023

Conversation

brunocodutra
Copy link
Owner

@brunocodutra brunocodutra commented Nov 18, 2023

Callgrind

valgrind --tool=callgrind --dump-instr=yes --collect-jumps=yes --cache-sim=yes --branch-sim=yes <cli> <<EOF go nodes 100000 EOF

==721543== Callgrind, a call-graph generating cache profiler
==721543== Copyright (C) 2002-2017, and GNU GPL'd, by Josef Weidendorfer et al.
==721543== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==721543== Command: engines/base
==721543== 
--721543-- warning: L3 cache found, using its data for the LL simulation.
==721543== For interactive control, run 'callgrind_control -h'.
info score cp 34 pv d2d4 g8f6 c2c4 e7e6 b1c3 d7d5 c1g5 c7c5 c4d5 c5d4 d1d4
bestmove d2d4
==721543== 
==721543== Events    : Ir Dr Dw I1mr D1mr D1mw ILmr DLmr DLmw Bc Bcm Bi Bim
==721543== Collected : 1725624658 597042803 301171008 13332658 51602269 12251903 4121 32416 267142 91463315 4906106 3406255 2064230
==721543== 
==721543== I   refs:      1,725,624,658
==721543== I1  misses:       13,332,658
==721543== LLi misses:            4,121
==721543== I1  miss rate:          0.77%
==721543== LLi miss rate:          0.00%
==721543== 
==721543== D   refs:        898,213,811  (597,042,803 rd + 301,171,008 wr)
==721543== D1  misses:       63,854,172  ( 51,602,269 rd +  12,251,903 wr)
==721543== LLd misses:          299,558  (     32,416 rd +     267,142 wr)
==721543== D1  miss rate:           7.1% (        8.6%   +         4.1%  )
==721543== LLd miss rate:           0.0% (        0.0%   +         0.1%  )
==721543== 
==721543== LL refs:          77,186,830  ( 64,934,927 rd +  12,251,903 wr)
==721543== LL misses:           303,679  (     36,537 rd +     267,142 wr)
==721543== LL miss rate:            0.0% (        0.0%   +         0.1%  )
==721543== 
==721543== Branches:         94,869,570  ( 91,463,315 cond +   3,406,255 ind)
==721543== Mispredicts:       6,970,336  (  4,906,106 cond +   2,064,230 ind)
==721543== Mispred rate:            7.3% (        5.4%     +        60.6%   )
==722425== Callgrind, a call-graph generating cache profiler
==722425== Copyright (C) 2002-2017, and GNU GPL'd, by Josef Weidendorfer et al.
==722425== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==722425== Command: engines/dev
==722425== 
--722425-- warning: L3 cache found, using its data for the LL simulation.
==722425== For interactive control, run 'callgrind_control -h'.
info score cp 34 pv d2d4 g8f6 c2c4 e7e6 b1c3 d7d5 c1g5 c7c5 c4d5 c5d4 d1d4
bestmove d2d4
==722425== 
==722425== Events    : Ir Dr Dw I1mr D1mr D1mw ILmr DLmr DLmw Bc Bcm Bi Bim
==722425== Collected : 1663235388 578141615 293224960 23009976 52763542 13670086 4463 32763 267349 89660959 4901258 3717656 2217967
==722425== 
==722425== I   refs:      1,663,235,388
==722425== I1  misses:       23,009,976
==722425== LLi misses:            4,463
==722425== I1  miss rate:          1.38%
==722425== LLi miss rate:          0.00%
==722425== 
==722425== D   refs:        871,366,575  (578,141,615 rd + 293,224,960 wr)
==722425== D1  misses:       66,433,628  ( 52,763,542 rd +  13,670,086 wr)
==722425== LLd misses:          300,112  (     32,763 rd +     267,349 wr)
==722425== D1  miss rate:           7.6% (        9.1%   +         4.7%  )
==722425== LLd miss rate:           0.0% (        0.0%   +         0.1%  )
==722425== 
==722425== LL refs:          89,443,604  ( 75,773,518 rd +  13,670,086 wr)
==722425== LL misses:           304,575  (     37,226 rd +     267,349 wr)
==722425== LL miss rate:            0.0% (        0.0%   +         0.1%  )
==722425== 
==722425== Branches:         93,378,615  ( 89,660,959 cond +   3,717,656 ind)
==722425== Mispredicts:       7,119,225  (  4,901,258 cond +   2,217,967 ind)
==722425== Mispred rate:            7.6% (        5.5%     +        59.7%   )

SPRT

cutechess-cli -sprt elo0=0 elo1=10 alpha=0.05 beta=0.05 -games 2 -rounds 2000 -openings file=engines/openings-6ply-1000.pgn plies=6 policy=round -concurrency 4 -ratinginterval 10 -resultformat wide -recover -engine conf=dev -engine conf=base -each tc=3+0.025

Score of dev vs base: 647 - 586 - 847  [0.515] 2080
...      dev playing White: 353 - 273 - 413  [0.538] 1039
...      dev playing Black: 294 - 313 - 434  [0.491] 1041
...      White vs Black: 666 - 567 - 847  [0.524] 2080
Elo difference: 10.2 +/- 11.5, LOS: 95.9 %, DrawRatio: 40.7 %
SPRT: llr 2.96 (100.7%), lbound -2.94, ubound 2.94 - H1 was accepted

Gauntlet

cutechess-cli -tournament gauntlet -games 2 -rounds 1500 -openings file=engines/openings-6ply-1000.pgn plies=6 policy=round -concurrency 5 -ratinginterval 10 -resultformat wide -recover -engine conf=dev -engine conf=Nalwald-18 -engine conf=Counter-5.0 -engine conf=StockNemo-5.7 -each tc=3+0.025

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw 
   0 dev                           -76       6    9000    2151    4079    2770   3536.0   39.3%   30.8% 
   1 Nalwald-18                    101      10    3000    1427     577     996   1925.0   64.2%   33.2% 
   2 Counter-5.0                    77      11    3000    1422     767     811   1827.5   60.9%   27.0% 
   3 StockNemo-5.7                  49      10    3000    1230     807     963   1711.5   57.0%   32.1%

Copy link

codecov bot commented Nov 18, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (c054e97) 91.62% compared to head (aaa0fa6) 88.37%.

Files Patch % Lines
lib/util/saturating.rs 92.85% 2 Missing ⚠️
lib/chess/position.rs 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #568      +/-   ##
==========================================
- Coverage   91.62%   88.37%   -3.25%     
==========================================
  Files          39       39              
  Lines        1170     1179       +9     
==========================================
- Hits         1072     1042      -30     
- Misses         98      137      +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@brunocodutra brunocodutra force-pushed the codegen branch 4 times, most recently from 3deec33 to 1a733d3 Compare November 21, 2023 23:29
@brunocodutra brunocodutra marked this pull request as ready for review November 27, 2023 19:18
@brunocodutra brunocodutra merged commit cded34e into master Nov 27, 2023
14 of 15 checks passed
@brunocodutra brunocodutra deleted the codegen branch November 27, 2023 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant