from Hacker News

Finding surprising moves in chess games

by mdp on 12/14/16, 12:47 AM with 35 comments

  • by Matetricks on 12/14/16, 1:30 AM

    If I understand your code correctly in analyse_evaluations, you're defining a "surprising move" as a move that has a large change in valuation when it's considered at a higher depth. So if a "human" (really Stockfish at depth 5) evaluates a move as +1 and a "computer" (Stockfish now at depth 11) evaluates the move as +5, the move is surprising.

    This is pretty interesting, but I'm not sure if it fully captures all the nuances of what a surprising move is. You might be able to classify a move as tactically surprising if it becomes clear after depth 7 that the ending position is favorable. However, in my opinion truly surprising moves are ones that carry plans that I haven't even considered. Hence, this methodology doesn't capture moves that are positionally surprising as there wouldn't be such a drastic change in evaluation at different depths. I'm not sure where you would start to figure that one out though :)

    That being said this is really cool work!

  • by al_biglan on 12/14/16, 2:23 AM

    This seems a perfect candidate to test with some well known test games.

    Fischer's 17 ... Be6, for example in the Game of the Century

    Or 15 ...Nf2 (again Fischer and Byrne) http://www.chessgames.com/perl/chessgame?gid=1008419&kpage=1

    It might be interesting to see how many blunders this identifies compared with traditional analysis. (Looking at the "surprising bad move" vs "surprising good move")

    In any case, very interesting work. Others have some good ideas for additional checks/considerations, and it would be interesting to see how this evolves!

  • by schoen on 12/14/16, 4:42 AM

    Note that Stockfish, which this project uses, just won the TCEC computer chess championship.

    https://en.wikipedia.org/wiki/TCEC

    There are other championships out there, but this is an indication that Stockfish is among the very strongest chess-playing entities in the world.

  • by CYHSM on 12/14/16, 2:21 PM

    Hi! I'm the author of this little library. Somebody apparently took it from my reddit thread (https://www.reddit.com/r/chess/comments/5i54lu/how_to_quanti...) and linked it here. Thank you for doing that, i haven't thought this would interest so many people!

    I hope i can answer some of your questions given more time but I have to say there are some very cool suggestions here and in the reddit thread which i have to keep in mind.

  • by billforsternz on 12/14/16, 6:41 AM

    I've been thinking about adding a game-analyse feature to my Tarrasch Chess GUI https://triplehappy.com. I was toying with the idea of using stockfish plus a poor engine as opposed to stockfish deep plus stockfish shallow. I was actually going to use my own Tarrasch Toy engine (about 1500 elo) as a reasonable proxy of a poor human player. My engine has fixed depth, so when set to very low depth will routinely just do the obvious thing and fall into simple fork traps etc., which would make it possible to auto generate simple explanatory variations like "Not $bad_move_played_by_my_engine because of $refutation_variation_pointed_out_by_stockfish."

    Like you I've noticed that often when looking at Engine Analysis the obvious human move doesn't appear as one of the first choices, which usually means there is an opportunity to get the machine to show you some tactics.

    So essentially your idea modulated by my assumption it would be difficult to get Stockfish to play sufficiently badly for stockfish deep plus stockfish shallow.

    But anyway, congrats to you for actually getting there first and making the idea work.

  • by dsjoerg on 12/14/16, 4:11 AM

    Really cool idea and interesting visualization! I am working on similar / related stuff if you'd like to collaborate further. dsjoerg at gmail.com. I participated in https://www.kaggle.com/c/finding-elo

  • by pk2200 on 12/14/16, 1:53 AM

    You might also add the condition that best-move(deep-search) != best-move(shallow-search). IMO this is what really makes a move surprising - it looks bad, but is good.

    Btw, I've never seen Rxd4!! from Kasparov-Topalov described as a "natural" move. I'm guessing the author meant "unnatural". :)

  • by vinchuco on 12/14/16, 1:47 AM

    I wish it had examples! I'd be more interested in a move that complicates the game. In fact, what is the most complicated position in chess? (Legal or not)