Creating an Engine Opening Book

        An opening book is basically a database and the first thing in creating your own is to decide what openings you want in your book  and, most importantly, what games you will use to select the moves.  You don’t want your engine being too predictable or worse yet, playing bad moves. Another option is to create an opening book containing games of a specific opening you wish to study.
        Most people define ‘opening theory’ as the openings and variations played by strong GM’s.  This is the problem with making opening books from databases with games by low rated players; unless you have filtered the games you can get bad lines played by lower rated players and the statistics will be misleading.  There is also the problem that opening books are compiled from games that were played in the past and new developments could have changed ‘theory.’
       Most opening books are created from games by 2400+ players but the question arises, why not just compile the book from games played only by super-GM’s? One problem is that there won’t be any offbeat or rarely played openings in the book because guys like Carlsen, Kramnik and Anand don’t play things like the Bird Opening.  Personally, in my opening book I don’t have any offbeat openings; my book is used exclusively for correspondence chess and most of my games are against higher rated players and offbeat lines are rare.  I figure that if somebody does play a line that isn’t in my book, I can easily switch to another, or better yet, search my database for games with those lines.
       Commercial engines have huge and highly optimized opening books but I prefer to have my games played by highly rated correspondence players because 1) they are strong analysts and 2) they have most likely heavily analyzed their opening lines with an engine.  Also, because I play most all of my games on ICCF or LSS servers where engine use is allowed, I have included top level engine vs. engine games just in case an opponent is relying on his engine to select a move.  I want to know if it’s been played by an engine before and what the result was.  Of course there is nothing wrong with including games of 2600 or 2700 plus players in your book if you want.
       Once you have compiled a book, the next question is what lines from that book should the engine play? Should it choose random variations from the book or should it rely on the statistical analysis of a large number of games based on the outcome?  The real problem here is that random selection could result in playing a line that resulted in a loss while whether a game is won or lost does not necessarily depend on the variation but could be the result of mistakes later on.  In the latter case, given a large enough number of games the statistical analysis could be useful.
       The most important numbers you will see in the Fritz opening book window are: N, %, Av and Perf.  For example, using Tartajubow’s Super Opening book, a recent game opened:

Sicilian: Kan Variation: 5 Bd3 1.e4 c5 2.Nf3 e6 3.d4 cxd4 4.Nxd4 a6

      N is the number of games in the opening book in which a move was played. % is the score by White.  Av is the average rating of the players.  Note that if a player’s rating isn’t given in the game score a default rating of 2200 is assigned.  Perf means that if a player’s rating was the ‘Av’ and he played the number of games shown in the ‘N’ column his new rating would be as shown in the ‘Perf’ column.  From the above you can see my book has 68 games where 5.Bd3 was played and the average rating of the players was 2573 and White scored 53.7%.  So, if a 2573 rated player played 68 games with 5.Bd3 his performance rating would be 2615.  Of course all that is based only on the games included in the DB and does not mean if you choose 5.Bd3 your results will be the same or that if there were 300 games in the db the outcome would be the same, but it does serve as a guideline that may help in deciding which move to play.

       In the above example the Fritz 12 opening book does not even show 4…a6 as a choice for Black, so it does not list any moves for White after Black’s 4…a6.  What you would have to do is rely on Fritz’ suggestion for your next move.  I let Fritz 12 run for several minutes and it came up with 5.Nc3 Qc7 6.Qf3 Nf6 7.Be2 Bd6 8.Be3 Nc6 9.0–0–0 Be5 10.g4 h6 11.Kb1 0–0 and believed White was about 3/4 of a P ahead.  All that is pretty meaningless because in its analysis Fritz did not recognize that after 5.Nc3 it was actually back in a heavily booked line.  The move I actually played was 5.Bd3 and the Fritz opening book still had no move for Black in its book.  However, after 5…Bc5 it recognized it was back in the book and listed four possibilities for White. A better solution to the above situation is to do a position search of the DB which would reveal any games that had reached the position after 4…a6 and you could then examine those games for the best continuation.

       It’s not my intention to discuss the technical details about how one creates an opening book as that information is widely available by people better qualified than I am.  I simply want to show that creating your own opening books can have some advantages. 

       Of course it’s always a good idea to keep the book updated.  Where to get games?  I download correspondence games from places like ICCF, LSS and various engine sites.  After that I sort and delete all games by players below a certain level (what Elo rating you want as a cutoff point is your call).  The next thing I do is only select games from recent years.  I don’t want games played 5-6 years ago because engines have greatly increased in strength since then.  You can also do some research on recent innovations in your favorite lines and add them to your opening book.  In the end, it is possible to end up with your own book that is better suited to your needs than the one that came with the engine.

Correspondence games can be downloaded at: ICCF and IECG

Engine games can be downloaded HERE.  These games are especially good because they are not blitz games; they were played at a tournament T/L of 40/2.

So, create your own specialty opening books and get the drop on you correspondence opponents or use it to study your favorite opening.