from Hacker News

GPT-3.5 passed yet another Theory of Mind test

by izzygonzalez on 2/20/23, 9:06 PM with 50 comments

  • by gnulinux on 2/21/23, 12:31 AM

    The answers aren't right at all... Answer to question 4 is clearly bogus as another commenter (Dfiesl) pointed out. But question 5 is also wrong. It's not unclear, from the conversation we can deduce that Ana thought that Maria is pregnant, otherwise she wouldn't have said it, unless she intentionally wants to make Maria uncomfortable, which is an unusual set of circumstances. What's more is, that possibility would be inconsistent with the answer to Q4 ("trying to make conversation").

    Test failed?

  • by RcouF1uZ4gsC on 2/20/23, 11:16 PM

    Actually, ChatGPT might be useful for actually testing the theory of mind. The philosophers were always working with an N of 1 (with respect to language) when they devised these tests. It is real easy to overfit a test if you have limited samples.

    Chat GPT is actually a good test as to which parts of the theory of mind are actually BS.

  • by Dfiesl on 2/20/23, 11:03 PM

    Seems like it got question 4 wrong... Who implies someone is pregnant to make them feel good? You imply someone is pregnant because they appear pregnant.

  • by kelseyfrog on 2/20/23, 11:12 PM

    It just predicts the next word.

  • by mensetmanusman on 2/21/23, 12:51 AM

    The answers to preëxisting theory of mind questions are stored in the graph network in a compressed sort of way, so I’m not surprised.