from Hacker News

Are LLMs able to play the card game Set?

by vnglst on 2/15/25, 10:28 AM with 31 comments

• by vnglst on 2/15/25, 10:28 AM

  Set is a card game where players have to identify sets of three cards from a layout of 12. Each card features a combination of four attributes: shape, color, number, and shading. A valid set consists of three cards where each attribute is either the same on all three cards or different on each. The goal is to find such sets quickly and accurately.

  Though this game is a solved computer problem — easily tackled by algorithms or deep learning — I thought it would be interesting to see if Large Language Models (LLMs) could figure it out.

• by Corence on 2/18/25, 11:18 PM

  FYI: Card 8's transcription is different than the image. In the image 5, 8, 12 is a Set but the transcription says Card 8 only has 2 symbols which removes that Set.

• by RheingoldRiver on 2/19/25, 12:24 AM

  Woah, what's going on?? I've always played Set with stripey cards, is this a custom deck or did they change it at some point???

  This is wildly disconcerting to me

• by bhouston on 2/19/25, 1:57 AM

  I noticed that LLM at least at the Claude and OpenAI 4o level can not play tic tac toe and win against a competent opponent. They make illogical moves.

  Interestingly, they can write a piece of code to solve Tic Tac Toe perfectly without breaking a sweat.

• by spuz on 2/19/25, 1:13 AM

  Since you can train an LLM to play chess from scratch, I would not be surprised if you could also train one to play Set. I might experiment with it tomorrow.

  https://adamkarvonen.github.io/machine_learning/2024/01/03/c...

• by zdw on 2/19/25, 1:57 AM

  Get them to play Fluxx, and we'll be talking...

  (this one, where ever changing rules is part of the game: https://www.looneylabs.com/games/fluxx )

• by James_K on 2/19/25, 12:10 AM

  I am increasingly concerned that these new reasoning models are thinking.