Fischer 960 (FRC) Engine Gauntlets

Komodo 11.2 - Stockfish 8

First round: Hyperbullet 10+0.1, 1 thread, AMD Opteron 2,4Ghz, 128MB Hash

(komodo depth in start pos: 12-14 sf:12-16)

Score of Komodo-11.2 vs Stockfish-8: 266 - 454 - 424  [0.418] 1144
Elo difference: -57.62 +/- 16.05
White Score  : 52.4 %
     Program            Elo      +      -   Games   Score   Av.Op.  Draws

   1 Stockfish-8    : 2329.1    0.0    0.0  1144    58.2 %   2271   37.1 %
   2 Komodo-11.2    : 2270.9    0.0    0.0  1144    41.8 %   2329   37.1 %

On my machine with longer timecontrol (eg 180+2), faster machine and 9 threads it looks to me, that komodo plays better.
So I tried to increase tc and added an cpu, at least the elo different should get smaller...

Second round: Bullet 60+1, 2 threads, AMD Opteron 2,4GHz, 512MB Hash

(komodo depth in start pos: 16-18 sf:16-21)

  White Score  : 53.4 %
  Program            Elo       +      -    Games   Score   Av.Op.  Draws
  1 Stockfish-8    : 2315.3    0.0    0.0  1000    54.4 %   2285   50.5 %
  2 Komodo-11.2    : 2284.7    0.0    0.0  1000    45.6 %   2315   50.5 %

Ok, much better, the diffenece is getting smaller, but still not what I like to see ;) So more time and threads:

Third round: Bullet 120+1, 4 threads, AMD Opteron 2,4GHz, 2048MB Hash

(komodo depth in start pos: 16-20 sf:19-21)

Score of Komodo-11.2 vs Stockfish-8: 210 - 251 - 539  [0.479] 1000
Elo difference: -14.25 +/- 14.61
White Score  : 55.1 %
   Program            Elo       +      -    Games   Score   Av.Op.  Draws
   1 Stockfish-8    : 2307.2    0.0    0.0  1000    52.0 %   2293   53.9 %
   2 Komodo-11.2    : 2292.8    0.0    0.0  1000    48.0 %   2307   53.9 %

Ok, its getting closer!
Lets see, how much komodo gains with Contempt=0. I expect them to be pretty equal, maybe the draw rate goes up?

Komodo 11.2 Contempt=0, Bullet 120+1, 4 threads, AMD Opteron 2,4GHz, 2048MB Hash

Score of Komodo-11.2 vs Stockfish-8: 184 - 223 - 593  [0.480] 1000
Elo difference: -13.56 +/- 13.72
White Score  : 56.0 %
  1 Stockfish-8    : 2306.8    0.0    0.0  1000    52.0 %   2293   59.3 %
  2 Komodo-11.2    : 2293.2    0.0    0.0  1000    48.0 %   2307   59.3 %

Hmm, Contempt=0 doesnt gain anything, it only raises the draw rate!

Komodo 11.2 Contempt=0, Blitz 180+2, 4 threads, AMD Opteron 2,4GHz, 2048MB Hash

Score of Komodo-11.2 vs Stockfish-8: 170 - 226 - 604  [0.472] 1000
Elo difference: -19.48 +/- 13.53
   White Score  : 54.1 %
  1 Stockfish-8    : 2309.8    0.0    0.0  1000    52.8 %   2290   60.4 %
  2 Komodo-11.2    : 2290.2    0.0    0.0  1000    47.2 %   2310   60.4 %

Hmm, komodo doesnt get better, even with longer tc

For fun, I just started an round of:

McBrain v2.2 - SF 8, Hyperbullet 10+0.1, 1 thread, on my i7-4930K CPU @ 3.40GHz, 128MB Hash

(depth 14-17 for both)

Score of McBrain-22 vs Stockfish-8: 234 - 134 - 632  [0.550] 1000
Elo difference: 34.86 +/- 13.00

White Score  : 54.8 %
Num. Name            games   score 
   0 McBrain-22       1000     550 
   1 Stockfish-8      1000     450 
Rank Name          Elo    +    - games score oppo. draws 
   1 McBrain-22     14   16   16  1000   55%   -14   63% 
   2 Stockfish-8   -14   16   16  1000   45%    14   63%

Impressive draw rate ^^ (but its a SF clone...)
But that means, mcbrain is the one to catch...

McBrain v2.2 - Komodo 11.2 Contempt=0, Hyperbullet 10+0.1, 1 thread, on my i7-4930K CPU @ 3.40GHz, 128MB Hash

(depth komodo 12-15, mcbrain 12-17)
should be around 70 difference...

  Score of McBrain-22 vs Komodo-11.2: 373 - 194 - 433  [0.590] 1000
  Elo difference: 62.87 +/- 16.26

  White Score  : 51.6 %
  Num. Name            games   score 
   0 McBrain-22       1000   589.5 
   1 Komodo-11.2      1000   410.5 
  Rank Name          Elo    +    - games score oppo. draws 
   1 McBrain-22     28   17   17  1000   59%   -28   43% 
   2 Komodo-11.2   -28   17   17  1000   41%    28   43%

lets see how this is with dynamism=80. I expect the draw rate to go up, if komodo would win more... i dont know

McBrain v2.2 - Komodo 11.2 Contempt=0, Dynamism=80, Hyperbullet 10+0.1, 1 thread, on my i7-4930K CPU @ 3.40GHz, 128MB Hash

(depth komodo 12-15, mcbrain 12-17)

Score of McBrain-22 vs Komodo-11.2: 465 - 173 - 362  [0.646] 1000
Elo difference: 104.49 +/- 17.50
Num. Name            games   score 
   0 McBrain-22       1000     646 
   1 Komodo-11.2      1000     354 
Rank Name          Elo    +    - games score oppo. draws 
   1 McBrain-22     48   18   18  1000   65%   -48   36% 
   2 Komodo-11.2   -48   18   18  1000   35%    48   36%

Oh, dynamism 80 seems like a bad idea...
The difference between a fresh CFish and mcbrain isnt realy noticable:

CFish-120817 - McBrain v2.2, Hyperbullet 10+0.1, 1 thread, on my i7-4930K CPU @ 3.40GHz, 128MB Hash

Score of McBrain-22 vs CFish-120817: 152 - 185 - 663  [0.483] 1000
Elo difference: -11.47 +/- 12.48

This all (except the dynamism experiement) piped to ordo:

   # PLAYER          :  RATING  POINTS  PLAYED   (%)
   1 CFish-120817    :  2333.3   516.5    1000    52
   2 McBrain-22      :  2321.8  1623.0    3000    54
   3 Stockfish-8     :  2286.6   450.0    1000    45
   4 Komodo-11.2     :  2258.3   410.5    1000    41