from Hacker News

Ask HN: Is AI a glorified memory machine?

by thallukrish on 8/24/23, 5:25 AM with 5 comments

If every problem is bounded by a bunch of data points on a multi-dimensional plane, and some sort of clustering is done on them that reflects the problem domain, then isn't running through the samples during training equivalent to memorizing them and associating them and allowing for a small margin of error making it look like the model is able to generalize. In short, doesn't that just boil down to how good we made our samples and setting the whole process in a way to memorize those samples ?

  • by Jack000 on 8/24/23, 8:09 PM

    As Yann Lecun said, “in high dimension, there is no such thing as interpolation. In high dimension, everything is extrapolation.”

    It’s trivial to come up with a prompt that doesn’t exist in the dataset. To generalize, the model cannot memorize.

  • by WallyFunk on 8/24/23, 4:26 PM

    (Current) AI is a glorified stochastic parrot[0]. Randomness != Reason.

    [0] https://en.wikipedia.org/wiki/Stochastic_parrot

  • by akasakahakada on 8/24/23, 7:53 AM

    TB of original materials condensed into a few GB of neural network weight. If this is memorization, I would like it to be implemented into archive.org.

  • by bpiche on 8/24/23, 5:43 AM

    Bruh