Leela Chess Zero (abbreviated as LCZero, lc0) is a free, open-source, and deep neural network–based chess engine and volunteer computing project. Development has been spearheaded by programmer Gary Linscott, who is also a developer for the Stockfish chess engine. Leela Chess Zero was adapted from the Leela Zero Go engine,[1] which in turn was based on Google's AlphaGo Zero project.[2] One of the purposes of Leela Chess Zero was to verify the methods in the AlphaZero paper as applied to the game of chess.
 | |
| Original author(s) | Gian-Carlo Pascutto, Gary Linscott |
|---|---|
| Developer(s) | Gary Linscott, Alexander Lyashuk, Folkert Huizinga, others |
| Initial release | 9 January 2018 |
| Stable release | v0.31.1
/ 11 August 2024 |
| Repository | |
| Written in | C++ |
| Operating system | Windows, Mac, Linux, Ubuntu, Android |
| Type | Chess engine |
| License | GPL-3.0-or-later |
| Website | lczero |
Like Leela Zero and AlphaGo Zero, Leela Chess Zero starts with no intrinsic chess-specific knowledge other than the basic rules of the game.[1] Leela Chess Zero then learns how to play chess by reinforcement learning from repeated self-play, using a distributed computing network coordinated at the Leela Chess Zero website.
As of June 2024[update], Leela Chess Zero has played over 2.5 billion games against itself, playing around 1 million games every day,[3] and is capable of play at a level that is comparable with Stockfish, the leading conventional chess program.[4][5]
History
editThe Leela Chess Zero project was first announced on TalkChess.com on January 9, 2018.[1][6] This revealed Leela Chess Zero as the open-source, self-learning chess engine it would come to be known as, with a goal of creating a strong chess engine.[7] Within the first few months of training, Leela Chess Zero had already reached the Grandmaster level, surpassing the strength of early releases of Rybka, Stockfish, and Komodo, despite evaluating orders of magnitude fewer positions due to its deep neural network in its evaluation function and its use of Monte Carlo tree search.
In December 2018, the AlphaZero team published a new paper in Science magazine revealing previously undisclosed details of the architecture and training parameters used for AlphaZero.[8] These changes were soon incorporated into Leela Chess Zero and increased both its strength and training efficiency.[9]
The work on Leela Chess Zero has informed the similar AobaZero project for shogi.[10]
The engine has been rewritten and carefully iterated upon since its inception, and since 2019[11] runs on multiple backends, allowing it to effectively utilize different types of hardware, both CPU and GPU.[12]
The engine can be configured to use different weights,[13] including even different architectures. This same mechanism of substitutable weights can also be used for alternative chess rules, such as for the Fischer Random Chess variant, which was done in 2019.[14]
Since 2022, it uses post-LN encoder-only transformers, similar to BERT. It represents a chessboard as a length-64 sequence of tokens.[15] The output of the network is another length-64 sequence of encoded tokens. A separate module is used to compute a logit for every possible move from one square (the source) to another (the target). The module takes as input two encoded tokens and outputs a single number (the logit for that move).[16]: Appendix A
Program and use
editThe method used by its designers to make Leela Chess Zero self-learn and play chess at above human level is reinforcement learning. This is a machine-learning algorithm, mirrored from AlphaZero used by the Leela Chess Zero training executable/binary code (called "binary") to maximize reward through self-play.[1][8] As an open-source distributed computing project, volunteer users run Leela Chess Zero to play hundreds of millions of games which are fed to the reinforcement algorithm.[3] In order to contribute to the advancement of the Leela Chess Zero engine, the latest non-release candidate (non-rc) version of the Engine as well as the Client must be downloaded. The Client is needed to connect to the current server of Leela Chess Zero, where all of the information from the self-play chess games are stored, to obtain the latest network, generate self-play games, and upload the training data back to the server.[17]
In order to play against the Leela Chess Zero engine on a machine, two components are needed: the engine binary and a network. (The engine binary is distinct from the client, in that the client is used as a training platform for the engine). The network contains Leela Chess Zero's evaluation function that is needed to evaluate positions.[17] Older networks can also be downloaded and used by placing those networks in the folder with the Lc0 binary.
Self-play Elo
editSelf-play Elo is used to gauge relative network strength to look for anomalies and general changes in network strength, and can be used as a diagnostic tool when Lc0 undergoes significant changes. Through test match games that are played with minimal temperature-based variation,[18] Lc0 engine clients test the most recent version against other recent versions of the same network's run, which is then sent to the training server to create an overall Elo assessment.
Standard Elo formulae are used to calculate relative Elo strength between the two players. More recent Self-play Elo calculations use match game results against multiple network versions to calculate a more accurate Elo value.
The Self-play approach has several consequences on gauging Lc0 Elo rating:
- Initial Cumulative Elo inflation in training runs differ drastically due to a-periodic gains in self-improvement and adversarial play.
- Measuring Elo relative to previous networks fails to measure general strength since networks are trained to anticipate and beat the predictions made by prior Lc0 networks rather than opponents outside the training domain. This is a type of overfitting measured most drastically when testing smaller networks.
- There is no direct one-to-one correlation between self-play Elo and the strength against Alpha-Beta engines, and no known correlation to strength against humans.
- Input training data has a significant effect on how a network will perform elo-wise against the next iteration.
- Cumulative Self-Play Elo does not have a universal conversion to conventional Human Elo due to inflation issues presented by adversarial play, and the reliance of the measure on time control. This holds true even when the engine runs on a standard set of initial conditions.
Cumulative Self-Play Elo inflation can be compared with other runs to gauge the lack of generality in gauging strength with pure cumulative self-play Elo. The Fischer Random Chess run Test 71.4 (named 714xxx nets), ranks at nearly 4000 cumulative self-play Elo 76 nets into its run (714076). The T60 (6xxxx) run 63000 net has a cumulative self-play Elo of around 2900. Pitting 714076 against net 63000 reveals 63000 clearly beats 714076 in head-to-head matches at most, if not all "fair" time controls. However, net 63000's Elo is significantly lower than its opponent. This supports the claim that Cumulative Self-Play Elo is not an objective measure of strength, nor is it a measure which allows one to linearly compare Lc0 network strength to human strength.
Setting up the engine to play a single node with --minibatch-size=1 and go nodes 1 for each played move creates deterministic play, and Self-Play Elo on such settings will always yield the same result between 2 of the same networks on the same start position—always win, always loss, or always draw. Self-play Elo is not reliable for determining strength in these deterministic circumstances.
Spinoffs
editIn season 15 of the Top Chess Engine Championship, the engine AllieStein competed alongside Leela. AllieStein is a combination of two different spinoffs from Leela: Allie, which uses the same evaluation network as Leela, but has a unique search algorithm for exploring different lines of play, and Stein, an evaluation network which has been trained using supervised learning based on existing game data featuring other engines (as opposed to the unsupervised learning which Leela uses). While neither of these projects would be admitted to TCEC separately due to their similarity to Leela, the combination of Allie's search algorithm with the Stein network, called AllieStein, is unique enough to warrant it competing alongside mainstream Lc0. (The TCEC rules require that a neural network-based engine has at least two unique components out of three essential features: The code that evaluates a network, the network itself, and the search algorithm. While AllieStein uses the same code to evaluate its network as Lc0, since the other two components are fresh, AllieStein is considered a distinct engine.)[19]
In early 2021, the LcZero blog announced Ceres, a new chess engine that uses LcZero networks. It implements Monte Carlo tree search as well as many novel algorithmic improvement ideas. Initial Elo testing showed that Ceres is stronger than Lc0 with the same network.[20]
Competition results
editIn April 2018, Leela Chess Zero became the first engine using a deep neural network to enter the Top Chess Engine Championship (TCEC), during season 12 in the lowest division, division 4.[21] Leela did not perform well: in 28 games, it won one, drew two, and lost the remainder; its sole victory came from a position in which its opponent, Scorpio 2.82, crashed in three moves.[22] However, it improved quickly. In July 2018, Leela placed seventh out of eight competitors at the 2018 World Computer Chess Championship.[23] In August 2018, it won division 4 of TCEC season 13 with a record of 14 wins, 12 draws, and 2 losses.[22][24] In Division 3, Leela scored 16/28 points, finishing third behind Ethereal, which scored 22.5/28 points, and Arasan on tiebreak.[25][22]
By September 2018, Leela had become competitive with the strongest engines in the world. In the 2018 Chess.com Computer Chess Championship (CCCC),[26] Leela placed fifth out of 24 entrants. The top eight engines advanced to round 2, where Leela placed fourth.[27][28] Leela then won the 30-game match against Komodo to secure third place in the tournament.[29][30] Leela participated in the "TCEC cup", an event in which engines from different TCEC divisions can play matches against one another. Leela defeated higher-division engines Laser, Ethereal and Fire before finally being eliminated by Stockfish in the semi-finals.[22]
In October and November 2018, Leela participated in the Chess.com Computer Chess Championship Blitz Battle.[31] Leela finished third behind Stockfish and Komodo.[32]
In December 2018, Leela participated in season 14 of the Top Chess Engine Championship. Leela dominated divisions 3, 2, and 1, easily finishing first in all of them. In the premier division, Stockfish dominated while Houdini, Komodo and Leela competed for second place. It came down to a final-round game where Leela needed to hold Stockfish to a draw with black to finish second ahead of Komodo. Leela managed this and therefore met Stockfish in the superfinal. In a back and forth match, first Stockfish and then Leela took three game leads before Stockfish won by the narrowest margin of 50.5–49.5.[22]
In February 2019, Leela scored its first major tournament win when it defeated Houdini in the final of the second TCEC cup. Leela did not lose a game the entire tournament.[22][33] In April 2019, Leela won the Chess.com Computer Chess Championship 7: Blitz Bonanza, becoming the first neural-network project to take the title.[34]
In the season 15 of the Top Chess Engine Championship (May 2019), Leela defended its TCEC cup title, this time defeating Stockfish in the final 5.5–4.5 (+2 =7 −1) after Stockfish blundered a seven-man tablebase draw.[35] Leela also won the Superfinal for the first time, scoring 53.5–46.5 (+14 −7 =79) versus Stockfish, including winning as both white and black in the same predetermined opening in games 61 and 62.[36][37]
Season 16 of TCEC saw Leela finish in third place in premier division, missing qualification for the superfinal to Stockfish and new deep neural network engine AllieStein. Leela did not suffer any losses in the Premier division, the only engine to do so, and defeated Stockfish in one of the six games they played. However, Leela only managed to score nine wins, while AllieStein and Stockfish both scored 14 wins. This inability to defeat weaker engines led to Leela finishing third, half a point behind AllieStein and a point behind Stockfish.[38] In the fourth TCEC cup, Leela was seeded first as the defending champion, which placed it on the opposite half of the brackets as AllieStein and Stockfish. Leela was able to qualify for the finals, where it faced Stockfish. After seven draws, Stockfish won the eighth game to win the match.[39]
In Season 17 of TCEC, held in January–April 2020, Leela regained the championship by defeating Stockfish 52.5–47.5, scoring a remarkable six wins in the final ten games, including winning as both white and black in the same predetermined opening in games 95 and 96.[40] It qualified for the superfinal again in Season 18, but this time was defeated by Stockfish 53.5–46.5.[41] In the TCEC Cup 6 final, Leela lost to AllieStein, finishing second.[42]
Season 19 of TCEC saw Leela qualify for the superfinal again. This time it played against a new Stockfish version with support for NNUE, a shallow neural network–based evaluation function used primarily for the leaf nodes of the search tree. Stockfish NNUE defeated Leela convincingly with a final score of 54.5–45.5 (+18 −9 =73).[43][44] Since then, Leela has repeatedly qualified for the superfinal, only to lose every time to Stockfish: +14 −8 =78 in Season 20, +19 −7 =74 in Season 21, +27 −10 =63 in Season 23, +20 −16 =64 in Season 24, +27 =50 -23 in Season 25, and +31 =52 -17 in Season 26.
Results summary
edit| Season | Division 4 | Division 3 | Division 2 | Division 1 | Division P | Infrafinal | Superfinal |
|---|---|---|---|---|---|---|---|
| 12 (2018) | 8th | – | – | – | – | – | – |
| 13 (2018) | 1st | 3rd | – | – | – | – | – |
| 14 (2018) | – | 1st | 1st | 1st | 2nd | – | 2nd |
| 15 (2019) | – | – | – | – | 2nd | – | 1st |
| 16 (2019) | – | – | – | – | 3rd | – | – |
| 17 (2020) | – | – | – | – | 1st | – | 1st |
| 18 (2020) | – | – | – | – | 2nd | – | 2nd |
| 19 (2020) | – | – | – | – | 2nd | – | 2nd |
| 20 (2020) | – | – | – | – | 1st | – | 2nd |
| 21 (2021) | – | – | – | – | 2nd | – | 2nd |
| 22 (2022) | – | – | – | – | 3rd | 3rd | – |
| 23 (2022) | – | – | – | – | 2nd | – | 2nd |
| 24 (2023) | – | – | – | – | 2nd | – | 2nd |
| 25 (2023) | – | – | – | – | 2nd | – | 2nd |
| 26 (2024) | – | – | – | – | 2nd | – | 2nd |
| Event | Result | Opponent | Score |
|---|---|---|---|
| Cup 1 (2018) | 3rd | - | – |
| Cup 2 (2019) | 1st | Houdini | 4.5–3.5 |
| Cup 3 (2019) | 1st | Stockfish | 5.5–4.5 |
| Cup 4 (2019) | 2nd | Stockfish | 3.5–4.5 |
| Cup 5 (2020) | 2nd | Stockfish | 1.5–2.5 |
| Cup 6 (2020) | 2nd | AllieStein | 1.5–2.5 |
| Cup 7 (2020) | 2nd | Stockfish | 1.5–2.5 |
| Cup 8 (2021) | 2nd | Stockfish | 3.5–4.5 |
| Cup 9 (2021) | 2nd | Stockfish | 1.5–2.5 |
| Cup 10 (2022) | 2nd | Stockfish | 4–6 |
| Cup 11 (2023) | 1st | Stockfish | 8.5–7.5 |
| Cup 12 (2023) | 2nd | Stockfish | 13.5–14.5 |
| Cup 13 (2024) | 2nd | Stockfish | 12.5–9.5 |
| Event | Year | Time Controls | Result | Ref |
|---|---|---|---|---|
| CCC 1: Rapid Rumble | 2018 | 15+5 | 3rd | [46] |
| CCC 2: Blitz Battle | 2018 | 5+2 | 3rd | [47] |
| CCC 3: Rapid Redux | 2019 | 30+5 | 2nd | [48] |
| CCC 4: Bullet Brawl | 2019 | 1+2 | 2nd | [49] |
| CCC 5: Escalation | 2019 | 10+5 | 2nd | [50] |
| CCC 6: Winter Classic | 2019 | 10+10 | 2nd | [51] |
| CCC 7: Blitz Bonanza | 2019 | 5+2 | 1st | [34] |
| CCC 8: Deep Dive | 2019 | 15+5 | 2nd | [5] |
| CCC 9: The Gauntlet | 2019 | 5+2, 10+5 | 3rd | [52] |
| CCC 10: Double Digits | 2019 | 10+3 | 3rd | [53] |
| CCC 11 | 2019 | 30+5 | 1st | [54] |
| CCC 12: Bullet Madness! | 2020 | 1+1 | 1st | [55] |
| CCC 13: Shapes | 2020 | 3+2, 5+5, 10+5, 15+5 | 1st | [56][57] |
| CCC 14 | 2020 | 15+5 | 1st | [58] |
| CCC Blitz 2021 | 2021 | 5+5 | 2nd | [59] |
| CCC Chess 960 Blitz | 2021 | 5+5 | 3rd | [60] |
| CCC 16: Rapid | 2021 | 15+3 | 2nd | [61] |
| CCC 16: Bullet | 2021 | 2+1 | 3rd | [62] |
| CCC 16: Blitz | 2022 | 5+5 | 3rd | [63] |
| CCC 17: Rapid | 2022 | 15+3 | 3rd | [64] |
| CCC 17: Bullet | 2022 | 2+1 | 3rd | [65] |
| CCC 17: Blitz | 2022 | 5+5 | 2nd | [66] |
| CCC 18: Rapid | 2022 | 15+3 | 2nd | [67] |
| CCC 19: Blitz | 2022 | 5+5 | 3rd | [68] |
| CCC 19: Rapid | 2022 | 15+3 | 2nd | [69] |
| CCC 19: Bullet | 2023 | 1+1 | 3rd | [70] |
| CCC 20: Blitz | 2023 | 3+2 | 2nd | [71] |
| CCC 20: Rapid | 2023 | 10+3 | 2nd | [72] |
| CCC 20: Bullet | 2023 | 1+1 | 4th | [73] |
| CCC 21: Blitz | 2023 | 3+2 | 3rd | [74] |
| CCC 21: Rapid | 2023 | 10+3 | 2nd | [75] |
| CCC 21: Bullet | 2023 | 1+1 | 5th | [76] |
| CCC 22: Blitz | 2024 | 3+2 | 3rd | [77] |
| CCC 22: Rapid | 2024 | 10+3 | 2nd | [78] |
| CCC 22: Bullet | 2024 | 1+1 | 4th | [79] |
| CCC 23: Blitz | 2024 | 3+2 | 3rd | [80] |
Notable games
edit- Leela vs Stockfish, CCCC bonus games, 1–0 Leela beats the world champion Stockfish engine despite a one-pawn handicap.
- Stockfish vs Leela Chess Zero – TCEC S15 Superfinal – Game 61 Leela completely outplayed Stockfish with black pieces in Trompovsky attack, Leela's eval went from 0.1 to −1.2 in one move, and Stockfish's eval did not go negative until 15 moves later.
References
edit- ^ a b c d "Leela Chess Zero: Full Elo Graph". Lczero.org. 7 March 2019. Archived from the original on 15 March 2023. Retrieved 7 March 2019.
- ^ "leela-zero". GitHub. Archived from the original on 16 October 2018. Retrieved 27 April 2018.
- ^ a b "LCZero". lczero.org. Archived from the original on 2023-09-08. Retrieved 2022-01-13.
- ^ "Lc0 Wins Computer Chess Championship, Makes History". Chess.com. 17 April 2019. Archived from the original on 2020-11-07. Retrieved 2019-05-29.
- ^ a b Pete (pete) (24 May 2019). "Stockfish Strikes Back, Tops Lc0 In Computer Chess Championship". Chess.com. Archived from the original on 2019-05-25. Retrieved 2019-05-29.
- ^ "Announcing lczero". TalkChess.com. Archived from the original on 12 June 2018. Retrieved 11 June 2018.
- ^ "Announcing lczero - TalkChess.com". www.talkchess.com. Archived from the original on 2018-06-12. Retrieved 2019-03-21.
- ^ a b Silver, David; Hubert, Thomas; Schrittwieser, Julian; et al. (6 December 2018). "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play" (PDF). Science. 362 (6419): 1140–1144. Bibcode:2018Sci...362.1140S. doi:10.1126/science.aar6404. PMID 30523106. S2CID 54457125. Archived (PDF) from the original on 1 September 2019. Retrieved 1 September 2019.
- ^ "AlphaZero paper, and Lc0 v0.19.1". 7 December 2018. Archived from the original on 14 February 2019. Retrieved 14 February 2019.
- ^ Kobayashi, Yuki (2019-09-15). "GitHub – kobanium/aobazero: Aoba Zero". GitHub. Archived from the original on 2020-06-27. Retrieved 2019-09-25.
- ^ "Backend configuration". lczero.org. 2019-04-17. Retrieved 2024-07-20.
- ^ "leela-chess-zero". GitHub. Archived from the original on 18 April 2020. Retrieved 11 May 2020.
- ^ "Getting Started". lczero.org. Retrieved 2024-07-20.
- ^ "Selfplay outputs chess960 "capture rook" moves". github.com. 2019-11-18. Retrieved 2024-07-20.
- ^ "Transformer Progress". lczero.org. 2024-02-28. Retrieved 2024-07-20.
- ^ Jenner, Erik; Kapur, Shreyas; Georgiev, Vasil; Allen, Cameron; Emmons, Scott; Russell, Stuart (2024-06-02). "Evidence of Learned Look-Ahead in a Chess-Playing Neural Network". arXiv:2406.00877 [cs.LG].
- ^ a b "The rewritten engine, originally for tensorflow. Now all other backends have been ported here.: LeelaChessZero/lc0". LCZero. 2019-03-20. Retrieved 2019-03-21.
- ^ Temperature is a technical concept that makes Lc0's play more random. High temperature effectively means Lc0 more often makes moves other than what it thinks are the best moves. This is a setting that is only meant to be used during training, to help encounter unfamiliar positions and therefore learn them.
- ^ "Allie+Stein, the new neural network based engine entering TCEC S15". 2 March 2019. Archived from the original on 27 February 2020. Retrieved 17 April 2020.
- ^ "Announcing Ceres". LcZero Blog. Archived from the original on 15 June 2024. Retrieved 26 July 2021.
- ^ "Breaking: Leela Chess Zero enters TCEC Season 12". Chessdom. 18 April 2018. Archived from the original on 10 July 2018. Retrieved 11 June 2018.
- ^ a b c d e f "Seasons 1-14 and TEC Cup 1-2 database". TEC Legacy. Chessdom. Archived from the original on 2019-12-07. Retrieved 2022-11-26.
- ^ "World Computer Chess Championship 2018". ICGA. Archived from the original on 27 June 2020. Retrieved 19 July 2018.
- ^ "Leela Chess Zero wins the gold medal in TCEC Div 4 | Chessdom". 11 August 2018. Archived from the original on 2019-03-21. Retrieved 2019-03-21.
- ^ "Ethereal chess engine wins the gold at TCEC Div 3 | Chessdom". 17 August 2018. Retrieved 2019-03-21.
- ^ "Computer Chess Championship with Top Engines". Chess.com. Archived from the original on 2018-11-02. Retrieved 2018-09-08.
- ^ "CCCC stage 2 ended. Leela 4th with a good performance! Stockfish undefeated!". LCZero Blog. 26 September 2018. Archived from the original on 27 September 2018. Retrieved 26 September 2018.
- ^ Cilento, Pete (26 September 2018). "Stockfish, Houdini Battle For Computer Chess Championship; Komodo vs Lc0 For 3rd". Chess.com. Archived from the original on 9 October 2018. Retrieved 9 October 2018.
- ^ "Leela wins the match series against Komodo and wins a Pawn odds game against Stockfish!". LCZero Blog. 3 October 2018. Archived from the original on 9 October 2018. Retrieved 9 October 2018.
- ^ Cilento, Pete (4 October 2018). "Stockfish Wins Computer Chess Championship Rapid; Lc0 Finishes 3rd". Chess.com. Archived from the original on 6 October 2018. Retrieved 9 October 2018.
- ^ Cilento, Pete (11 October 2018). "Computer Chess Championship Returns For Blitz Battle". Chess.com. Archived from the original on 22 November 2018. Retrieved 22 November 2018.
- ^ Cilento, Pete (19 November 2018). "Stockfish Wins Computer Chess Championship Blitz". Chess.com. Archived from the original on 22 November 2018. Retrieved 22 November 2018.
- ^ "Leela won the TCEC CUP!". LCZero Blog. 4 February 2019. Archived from the original on 12 February 2019. Retrieved 12 February 2019.
- ^ a b Cilento, Pete (17 April 2019). "Lc0 Wins Computer Chess Championship, Makes History". Chess.com. Archived from the original on 7 November 2020. Retrieved 18 April 2019.
- ^ "TCEC Cup 3 - Final game statistics". TCEC. Retrieved 2022-11-26.
- ^ "Lc0 won TCEC 15". LCZero Blog. 28 May 2019. Archived from the original on 28 May 2019. Retrieved 28 May 2019.
- ^ Högy, Kevin (2 June 2019). "A new age in computer chess? Lc0 beats Stockfish!". chess24. Archived from the original on 8 June 2019. Retrieved 25 June 2019.
- ^ "Season 16, Div P archive". Archived from the original on 27 June 2020. Retrieved 30 September 2019.
- ^ "TCEC Cup 4 archive". Archived from the original on 3 March 2020. Retrieved 18 November 2019.
- ^ "TCEC S17 SUper FInal report - Leela Chess Zero". lczero.org. Archived from the original on 2020-05-16. Retrieved 2020-05-30.
- ^ "TCEC Season 18 archive". Archived from the original on 4 September 2019. Retrieved 3 July 2020.
- ^ "TCEC Cup 6 Final". TCEC Chess. Archived from the original on 4 September 2019. Retrieved 23 July 2020.
- ^ "Season 19, Div SF archive". Retrieved 19 October 2020.
- ^ "Stockfish 12". Stockfish Blog. Archived from the original on 26 November 2020. Retrieved 19 October 2020.
- ^ a b "TCEC – Live Chess Broadcast". Archived from the original on 2019-09-23. Retrieved 2019-09-23.
- ^ Cilento, Pete (4 October 2018). "Stockfish Wins Computer Chess Championship Rapid; Lc0 Finishes 3rd". Chess.com. Archived from the original on 2018-10-06. Retrieved 2019-06-20.
- ^ Cilento, Pete (20 November 2018). "Stockfish Wins Computer Chess Championship Blitz". Chess.com. Archived from the original on 2018-11-22. Retrieved 2019-06-20.
- ^ Cilento, Pete (22 January 2019). "Stockfish Wins Rapid Computer Championship Over Lc0; Bullet Chess Next". Chess.com. Archived from the original on 2020-11-07. Retrieved 2019-06-20.
- ^ Cilento, Pete (31 January 2019). "Stockfish Wins Computer Chess Championship Bullet; 'Escalation' Next". Chess.com. Archived from the original on 2021-01-19. Retrieved 2019-06-20.
- ^ "CCC 5: Finals (10|5)". Chess.com. 2019-02-08. Archived from the original on 2022-12-10. Retrieved 2024-06-15.
- ^ Cilento, Pete (20 March 2019). "Computer Chess Championship Plays Blitz After Stockfish Defends Title". Chess.com. Archived from the original on 2020-11-11. Retrieved 2019-06-20.
- ^ Cilento, Pete (7 August 2019). "Stockfish Wins Computer Chess Championship As Neural Networks Play Catch-Up". Chess.com. Archived from the original on 10 August 2019. Retrieved 19 September 2019.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 19 September 2019.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 23 December 2019.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 23 January 2020.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 14 April 2020.
- ^ Doggers, Peter (18 April 2020). "Leela Chess Zero Beats Stockfish 106–94 In 13th Chess.com Computer Chess Championship". Chess.com. Archived from the original on 28 April 2020. Retrieved 24 April 2020.
- ^ "CCC14 Results". CCC Formats. Archived from the original on 23 February 2021. Retrieved 19 August 2020.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 February 2022.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 February 2022.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 February 2022.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 February 2022.
- ^ "Computer Chess Championship". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 February 2022.
- ^ "Computer Chess Championship CCC 17: Rapid". Chess.com. Archived from the original on 2 November 2018. Retrieved 30 March 2022.
- ^ "Computer Chess Championship CCC 17: Bullet". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 17: Blitz". Chess.com. Archived from the original on 2 November 2018. Retrieved 13 July 2022.
- ^ "Computer Chess Championship CCC 18: Rapid". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 19: Blitz". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 19: Rapid". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 19: Bullet". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 20: Blitz". Chess.com. Archived from the original on 2 November 2018. Retrieved 24 March 2023.
- ^ "Computer Chess Championship CCC 20: Rapid". Chess.com. Archived from the original on 2 November 2018. Retrieved 30 June 2023.
- ^ "Computer Chess Championship CCC 20: Bullet". Chess.com. Archived from the original on 2 November 2018. Retrieved 4 August 2023.
- ^ "Computer Chess Championship CCC 21: Blitz". Chess.com. Archived from the original on 2 November 2018. Retrieved 18 August 2023.
- ^ "Computer Chess Championship CCC 21: Rapid". Chess.com. Archived from the original on 2 November 2018. Retrieved 13 October 2023.
- ^ "Computer Chess Championship with Top Engines". Chess.com. Archived from the original on 2018-11-02. Retrieved 2023-12-03.
- ^ "CCC22 Blitz: Challenger Match". Chess.com. Retrieved 2024-09-03.
- ^ "CCC22 Rapid: Finals". Chess.com. Retrieved 2024-09-03.
- ^ "CCC22 Bullet: Semifinals". Chess.com. Retrieved 2024-09-03.
- ^ "CCC23 Blitz: Challenger Match". Chess.com. Retrieved 2024-11-01.