Wednesday, September 29, 2010

Engine Comparison

Engines are very strong at tactical analysis, but weaker in quiet positions where strategy is required.

The endgames of chess programs are often enhanced by the use of endgame tablebases. But to be really effective these tablebases require a tremendous amount of disk space. I advise, if one needs ending analysis, you use Shredder’s online 6-piece tablebase.

The most popular chess engine protocol under use and probably supporting the strongest engine play is UCI (universal chess interface). These UCI protocols are stronger than the old XBoard/WinBoard engines. The top engines on the SSDF rating list are

RANK ENGINE RATING

1 -Deep Rybka 3 x64 2GB Q6600 2,4 GHz 3213
2 -Naum 4.2 MP x64 2GB Q6600 2,4 GHz 3168
3 -Naum 4 x64 2GB Q6600 2,4 GHz 3142
4 -Hiarcs 13.1 2GB Q6600 2,4 GHz 3122
5 -Deep Shredder 12 x64 2GB Q6600 2,4 GHz 3117
6 -Deep Fritz 12 2GB Q6600 2,4 GHz 3110
7 -Deep Rybka 3 256MB Athlon 1200 MHz 3084
8 -Deep Fritz 11 2GB Q6600 2,4 GHz 3073
9 -Zappa Mexico II x64 2GB Q6600 2,4 GHz 3061
10 -Naum 3.1 x64 2GB Q6600 2,4 GHz 3045
11 -Deep Hiarcs 12 2GB Q6600 2,4 GHz 3034
12 -Deep Shredder 11 x64 2GB Q6600 2,4 GHz 3029
13 -Hiarcs 11.2 MP 2GB Q6600 2,4 GHz 3001
14 -Glaurung 2.2 x64 MP 2GB Q6600 2,4 GHz 3000
15 -Naum 4 256MB Athlon 1200 MHz 2992
16 -Shredder 12 256MB A1200 MHz 2980
17 -Deep Junior 10.1 2GB Q6600 2,4 GHz 2975
18 -Fritz 12 256MB A1200 MHz 2939
19 -Rybka 2.3.1 Arena 256MB Athlon 1200 MHz 2920
20 -Fritz 11 256MB Athlon 1200 MHz 2912


These ratings are based on engine vs. engine play. Engines that are designed to be run on multiple processors are denoted by the term "Deep."

Unless you are a titled CC player participating in top level CC tournaments you are not going to purchase 4 CPUs to run your engine.

There are many free engines available that, for the most part will meet all the requirements of the average player. One such engine that does not appear on the SSDF list is Houdini. In an unscientific experimental blitz match I conducted, Houdini easily defeated Fritz 12 and the free Rybka engine.

Chessbase is the world's premier supplier of engines. Their engines come with the Fritz interface which is a database that allows you to perform many advanced database operations. The Fritz interface also had the best printout of all the programs. Purchases from Chessbase also include a 1-year subscription to the site PlayChess.com.


Rybka is more conservative in its evaluations. For example, if Fritz or Shredder think White is better by 1.5 pawns, Rybka often only considers White to be up only 1.0 pawn. Shredder’s attacks are not as speculative as previous versions.

The developers of Junior claim they haven't developed Junior to have the highest rating, but have tried to enable humans to gain new insights and understanding in the game. Junior is a very aggressive engine and its particularly strong at considering compensation in the case of material imbalances. This makes Junior an interesting in exploring sacrificial attacks and sharp positions.

As GM john Nunn points out, "all are extremely strong tactically, while all show deficiencies in quiet positions"

The SSDF tests are mainly commercial engines. For those that are interested in free engines, you should checkout Kirill Kryukov’s site KCEC.

He has run tests on a variety of freeware engines. The top rated engines are: Fruit, Spike, Glaurung, Naum, Bright, Alaric, Scorpio, E.T. Chess, etc. Unfortunately, again, Houdini is not on his list.

In so far as I know, nobody has compared free engines against commercial engines. I can understand this. Producers of commercial engines, naturally, would not like seeing their engines losing to freeware. Freeware authors can’t afford to purchase several hundred dollars worth of software required to run engine vs. engine tournaments.

All chess players these days use engines to some extent. IMO they are of little value in improving your play with the exception of pointing out tactical errors. They do nothing to help you understand how to evaluate a position and it is precisely evaluating positions that distinguishes players of varying strengths. GM’s are better than the rest of us because they know how to correctly evaluate which moves are good and which ones aren’t. Engines are notoriously not very good in playing endings either.

As a serious correspondence player, I use Chessbase Light and paid for the upgraded version simply because I needed to be able to access databases containing more than the 8000 game limit so as to research openings and GM games to see how they played various types of middlegames. When it comes to actually studying chess, about all engines are good for is 1) trying out different moves to see how they compare. 2) playing over games while trying to guess the next move as a means of increasing your pattern recognition skills. 3) Researching games to see how GM’s played certain types of positions. Beyond that engine analysis won’t tell you why certain moves are better than others. The exception is that engine suppliers offer a wide variety of programs designed for helping you study various aspects of the game.

Personally, if I were going to study chess with the goal of improving, I would use a good chess book in conjunction with a freeware engine to play over the games and positions from the book.

8 comments:

  1. http://www.computerchess.org.uk/ccrl/4040

    CCRL (Computer Chess Rating Lists) website contains a comprehensive rating list encompassing both free and commercial chess engines.

    The most remarkable fact, IMHO, is that the free chess engine Stockfish is the 2nd stronger, being only below Rybka ( which is commercial ).

    It's really very strong, and it's free.

    ReplyDelete
  2. Thanks your your input! That's an impressive list...nearly 200 engines.

    ReplyDelete
  3. Well, I enjoy your blog a lot ! I follow it on my RSS Google Reader and it's very nice.

    You're welcome.

    ReplyDelete
  4. Hi! I am the guy who posted on the TobyTal conversation with Houdini. I actually do not play much, but enjoying studying over old games with the engines in the background. I like the Fritz function of "Explain All Moves". But when I look over GM analysis, I would have to say the engines that come closest to what the author suggests are Shredder, then Hiarcs Paderborn, then ProDeo. C

    ReplyDelete
  5. I enjoy playing over older games with an engine also. I had an older version of Shredder and liked the feature where it ran two engines and had an output giving what it rated as best. The feature was called "compare" or something.

    ReplyDelete
  6. Yes, the "Triple Brain". It still has it! Even back as far as Shredder 5.32 I was amazed at how often Shredder played "Human-Like". A fun thing about the Triple Brain is you can make exotic personalities which will sometimes find brilliant moves (but will be terrible in engine tournaments. Like you, I really appreciate the history of the game and go over classic GM games :-)

    ReplyDelete
  7. Another interesting ratings list is IPON:

    http://www.inwoba.de/

    ReplyDelete