Thursday, December 15, 2011

Engine Blitz Tournament

In view of the results of a recent post where Fritz 12 did quite well on a few positional tests I decided to run a 5 minute tournament using some of my favorite engines just to how well Fritz would do and the results were disappointing. 


I thought perhaps is might be due to the fact that at a fast time limit Fritz might be making tactical errors. Here’s what a quick review of Fritz’ games revealed:


Vs. Houdini (32bit) – Fritz won a R+B+P ending.  Well-played by Fritz
Vs. Spike – they castled on opposite wings. Fritz had a better position but didn’t follow up its K-side attack in the strongest way and threw away the advantage.  Spike gained a significant advantage but then misplayed its Q-side attack and the game was drawn in a blocked position. Neither engine played particularly well, both missing some strong moves.
Vs Critter – in a R and N ending where both side had 3P’s, Critter had 3P’s vs. 2P’s on the K-side and Fritz had a passed P on a7, the game looked to be drawn because Fritz’ R was stuck on a6 in front of its P.  Critter began a series of checks that should have lead to a draw when Fritz, in an effort to escape the checks, walked into a mate and had to sacrifice it’s N leaving Critter with a won ending. A serious tactical error by Fritz lost this game.
Vs. Fire xTreme Fritz’ K-side onslaught out of the opening didn’t lead anywhere and Fire xTreme had about a 1/2 P advantage and traded down to a R and P ending which Fritz lost.
Vs. Firebird in a fairly level position Fritz made a bad exchange of its B for a N, drifted into an inferior position and lost a R+N vs. R+B ending when Firebird got its R on the 7th rank.
Vs. Stockfish in an equal position Fritz voluntarily weakened it P structure the plopped a N on e5 only to find that after …f3 it had to play N/e5xNc6 and give Stockfish the open b-file and the game was all but over.
Vs. Ivanhoe – after an opening error it got an inferior position and walked into a pin that ultimately lost a piece.  Another tactical error.
All-in-all Fritz’ play was not very impressive.  Against Critter and Ivanhoe it made serious tactical errors and in the other games it lost, it always seemed to drift into inferior positions from which it couldn’t recover.

However, to be fair Fritz did do quite well on the brief strategic quiz when given enough time to think. On the SSDF rating list Fritz 12 stands in 21st place behind the deep versions of Rybka, Shredder, the deep version of Fritz, Junior and, what shall we call them?  The ‘shallow’ Naum, Hiarcs, Zappa and Glaurung.  It has an overwhelming score of 26.5 – 13.5 against Spike. On the CCRL 40/40 rating list Houdini (64bit) is in first place with Rybka, Critter and Stockfish so the placing of the latter two in my little experiment isn’t surprising. 



I’ve been impressed with Fire xTreme the few times I’ve used it in the past and think it may be time to investigate its possibilities a little closer.  I think its development has ended and it may no longer be available for download…at least I couldn’t find it listed anywhere.

Stockfish overlooked a nice tactical shot in this game.


5 comments:

  1. For fun I ran a blitz tournament with the engines you tried and a few others. I did not have Fritz so substituted Hiarcs. Rybka, Naum, Fire, and Ivanhoe are probably close because of their FRUIT roots. An older version of Ivanhoe was slightly higher in this short test. A Rybka personality that I made was third. One of these days I will have to let a similar tournament run for many more games.

    Rank Name Iv Iv Ry Cr Ry Ho Fi Na Cy Jo Kt Lo Li HI br Points S-B %
    1 Ivanhoe-BetaWH_79_w32-mate X ½ 1 ½ ½ 1 1 1 1 1 1 1 1 1 1 12.5 / 14 77.50 89.29%
    2 IvanHoe 9.47b w32 ½ X 1 0 ½ 1 0 1 1 1 1 1 1 1 1 11.0 / 14 65.00 78.57%
    3 Rybka 4 w32 [TrickyTahoma] 0 0 X 1 1 1 0 1 ½ 1 1 1 1 1 1 10.5 / 14 59.00 75.00%
    4 Critter 1.2 32-bit ½ 1 0 X 0 1 ½ 1 1 0 1 1 1 ½ 1 9.5 / 14 59.00 67.86%
    5 Rybka 4 w32 [endgame] ½ ½ 0 1 X 0 ½ ½ 1 ½ 1 1 1 1 1 9.5 / 14 53.75 67.86%
    6 Houdini 1.5a w32 0 0 0 0 1 X 1 1 0 1 1 1 1 1 1 9.0 / 14 45.50 64.29%
    7 Fire 1.5 xTreme w32 0 1 1 ½ ½ 0 X ½ 1 1 0 1 1 0 1 8.5 / 14 54.50 60.71%
    8 Naum 4.1 [test] 0 0 0 0 ½ 0 ½ X 1 1 1 1 1 1 1 8.0 / 14 35.50 57.14%
    9 Cyclone xTreme Rage 0 0 ½ 0 0 1 0 0 X 1 1 ½ 1 1 1 7.0 / 14 31.75 50.00%
    10 Jonny 4.00 0 0 0 1 ½ 0 0 0 0 X ½ 1 0 ½ 1 4.5 / 14 21.75 32.14%
    11 Ktulu 7.1 0 0 0 0 0 0 1 0 0 ½ X ½ 0 1 1 4.0 / 14 15.75 28.57%
    12 Loop 2007 32-Bit 0 0 0 0 0 0 0 0 ½ 0 ½ X 1 1 1 4.0 / 14 12.50 28.57%
    13 Little Goliath Nemesis 0 0 0 0 0 0 0 0 0 1 1 0 X 1 1 4.0 / 14 11.50 28.57%
    14 HIARCS 13.2 MP 0 0 0 ½ 0 0 1 0 0 ½ 0 0 0 X 1 3.0 / 14 15.50 21.43%
    15 bright-0.5c 0 0 0 0 0 0 0 0 0 0 0 0 0 0 X 0.0 / 14 0.00 0.00%


    105 out of 105 games played
    Level: 1 Minutes/Game

    105 out of 105 games played
    Level: 1 Minutes/Game

    ReplyDelete
  2. Thanks! Interesting. I’ve run a number of blitz tournaments and the results are never consistent. I think it probably has to do with the time limit. What I would really like to do is run a tournament where the engines had, say, 3-5 minutes per move because that would be more in line with the time actually allowed when analyzing a game. Unfortunately, I don’t have the time to dedicate the laptop to the amount of time it would take because a lot of engine games can be long and, therefore, quite time consuming.

    ReplyDelete
  3. I just started one at four minutes. I running it piecemeal so it will take a little while but I will post it when I am done. :-)

    ReplyDelete
  4. Great!! I think a tournament run at long time controls would be a significant contribution to the chess community. Maybe there would be other places where it could be posted, too!

    ReplyDelete
  5. Well I let the tournament run overnight as well so making progress. 281 games so far at 4 minutes. Seems to be holding from the last time I checked. Collapsing some test personalities I made, the order is 1) Ivanhoe 2) Rybka 4 3) Fire 4) HIARCS 5) Naum 6) Houdini and Rybka Winfinder last. Kind of strange as I have seen Houdini at the top of other Blitz lists. Could be as not enough games or Houdini does not do as well with the Shredder book (ran tournament on Shredder GUI with Shredder 10 book)

    Rank Name Iv Ry Ry Fi Ry Ry Ry HI Na Ry Na Ho Na Na Na Na Na Ry HI Ry Points S-B %
    1 IvanHoe 9.47b w32 XX 0 ½½ ½ 1½ 01 11 1 11 1 ½½ ½ 1 ½1 11 1 11 1 1 1 22.0 / 28 289.75 78.57%
    2 Rybka 4 w32 [TrickyTahoma] 1 XX 1½ 0½ ½½ ½½ 1½ 1 ½ ½½ ½ ½ 1 ½ ½ 1 1 11 11 11 20.0 / 28 249.75 71.43%
    3 Rybka 4 w32 [endgame] ½½ 0½ XX 00 0 ½ 1 11 1 1½ 1 1½ 11 ½ 1 1 1 1 1½ ½1 19.5 / 28 247.25 69.64%
    4 Fire 2.2 xTreme w32 ½ 1½ 11 XX ½0 ½½ 0½ ½ 1 1½ 1 0 ½ 1 ½1 ½ 11 ½½ 1 11 19.5 / 29 251.50 67.24%
    5 Rybka 4 w32 [crazy] 0½ ½½ 1 ½1 XX 0 ½ ½1 0 ½1 ½ 10 ½1 ½ 1 ½1 1 ½ 11 1 18.0 / 28 236.25 64.29%
    6 Rybka 4 w32 10 ½½ ½ ½½ 1 XX ½ ½½ 0 ½ ½½ 11 ½½ 1 ½ 01 1 ½ 11 1 17.5 / 28 234.00 62.50%
    7 Rybka 4 w32 [pawnplus] 00 0½ 0 1½ ½ ½ XX 1½ 0 ½1 1 00 ½ ½ 1 1 ½ 11 11 11 16.5 / 28 188.25 58.93%
    8 HIARCS 13.2 SP [Deeper] 0 0 00 ½ ½0 ½½ 0½ XX 0 ½1 ½ 1 ½ ½½ 11 1 11 1 1 11 16.0 / 28 174.75 57.14%
    9 Naum 4.1 [Tahoma-Smart!] 00 ½ 0 0 1 1 1 1 XX 0 00 11 ½½ ½½ 0½ ½1 ½1 1½ 1 1 15.5 / 28 185.25 55.36%
    10 Rybka 4 w32 [Tahoma] 0 ½½ 0½ 0½ ½0 ½ ½0 ½0 1 XX 1 ½ ½ 1 1 1 ½ 11 ½½ 11 15.5 / 28 180.00 55.36%
    11 Naum 4.1 [Tahoma-Senior] ½½ ½ 0 0 ½ ½½ 0 ½ 11 0 XX 0 10 ½0 11 11 10 01 1 1 15.0 / 28 182.25 53.57%
    12 Houdini 1.5a w32 ½ ½ 0½ 1 01 00 11 0 00 ½ 1 XX ½ 11 ½0 1 01 ½ 0 11 14.5 / 28 185.75 51.79%
    13 Naum 4.1 0 0 00 ½ ½0 ½½ ½ ½ ½½ ½ 01 ½ XX ½0 ½1 ½1 11 1 ½ 1 14.0 / 28 163.75 50.00%
    14 Naum 4.1 [test] ½0 ½ ½ 0 ½ 0 ½ ½½ ½½ 0 ½1 00 ½1 XX 0 ½½ ½1 11 ½. 1 13.5 / 28 164.25 48.21%
    15 Naum 4.1 [Vodka] 00 ½ 0 ½0 0 ½ 0 00 1½ 0 00 ½1 ½0 1 XX ½½ ½ 1 11 1 11.5 / 28 129.25 41.07%
    16 Naum 4.1 [Tahoma] 0 0 0 ½ ½0 10 0 0 ½0 0 00 0 ½0 ½½ ½½ XX ½½ 0½ 1 1 8.5 / 28 96.00 30.36%
    17 Naum 4.1 [TrickyTahoma] 00 0 0 00 0 0 ½ 00 ½0 ½ 01 10 00 ½0 ½ ½½ XX 1 ½½ 1 8.5 / 29 89.75 29.31%
    18 Rybka WinFinder 2.2 32-bit 0 00 0 ½½ ½ ½ 00 0 0½ 00 10 ½ 0 00 0 1½ 0 XX ½ 11 8.0 / 28 84.50 28.57%
    19 HIARCS 13.2 SP [Brute] 0 00 0½ 0 00 00 00 0 0 ½½ 0 1 ½ ½. 00 0 ½½ ½ XX 11 7.0 / 28 67.00 25.00%
    20 Rybka 4 w32 [PQ++] 0 00 ½0 00 0 0 00 00 0 00 0 00 0 0 0 0 0 00 00 XX 0.5 / 28 9.75 1.79%



    281 out of 380 games played
    Level: 4 Minutes/Game

    ReplyDelete