by alphadelphi on 4/2/25, 10:59 PM with 140 comments
by antirez on 4/5/25, 4:31 PM
So at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues:
1. Weakest ever LLM among the big labs with similar resources (and smaller resources: DeepSeek).
2. They say they are focusing on open source models, but the license is among the less open than the available open weight models.
3. LLMs and in general all the new AI wave puts CNNs, a field where LeCun worked (but that didn't started himself) a lot more in perspective, and now it's just a chapter in a book that is composed mostly of other techniques.
Btw, other researchers that were in the LeCun side, changed side recently, saying that now "is different" because of CoT, that is the symbolic reasoning they were blabling before. But CoT is stil regressive next token without any architectural change, so, no, they were wrong, too.
by gsf_emergency_2 on 4/5/25, 3:45 PM
LeCun, "Mathematical Obstacles on the Way to Human-Level AI"
Slide (Why autoregressive models suck)
by djoldman on 4/5/25, 4:58 PM
More interesting is their research work. JEPA is what LeCun is betting on:
https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...
by redox99 on 4/5/25, 5:41 PM
by csdvrx on 4/5/25, 3:57 PM
Many people believe that "wants" come first, and are then followed by rationalizations. It's also a theory that's supported by medical imaging.
Maybe the LLM are a good emulation of system-2 (their perfomance sugggest it is), and what's missing is system-1, the "reptilian" brain, based on emotions like love, fear, aggression, (etc.).
For all we know, the system-1 could use the same embeddings, and just work in parallel and produce tokens that are used to guide the system-2.
Personally, I trust my "emotions" and "gut feelings": I believe they are things "not yet rationalized" by my system-2, coming straight from my system-1.
I know it's very unpopular among nerds, but it has worked well enough for me!
by bitethecutebait on 4/5/25, 9:23 PM
by ejang0 on 4/5/25, 3:42 PM
by GMoromisato on 4/5/25, 4:22 PM
He wrote about Copycat, a program for understanding analogies ("abc is to 123 as cba is to ???"). The program worked at the symbolic level, in the sense that it hard-coded a network of relationships between words and characters. I wonder how close he was to "inventing" an LLM? The insight he needed was that instead of hard-coding patterns, he should have just trained on a vast set of patterns.
Hofstadter focused on Copycat because he saw pattern-matching as the core ability of intelligence. Unlocking that, in his view, would unlock AI. And, of course, pattern-matching is exactly what LLMs are good for.
I think he's right. Intelligence isn't about logic. In the early days of AI, people thought that a chess-playing computer would necessarily be intelligent, but that was clearly a dead-end. Logic is not the hard part. The hard part is pattern-matching.
In fact, pattern-matching is all there is: That's a bear, run away; I'm in a restaurant, I need to order; this is like a binary tree, I can solve it recursively.
I honestly can't come up with a situation that calls for intelligence that can't be solved by pattern-matching.
In my opinion, LeCun is moving the goal-posts. He's saying LLMs make mistakes and therefore they aren't intelligent and aren't useful. Obviously that's wrong: humans make mistakes and are usually considered both intelligent and useful.
I wonder if there is a necessary relationship between intelligence and mistakes. If you can solve a problem algorithmically (e.g., long-division) then there won't be mistakes, but you don't need intelligence (you just follow the algorithm). But if you need intelligence (because no algorithm exists) then there will always be mistakes.
by grandempire on 4/5/25, 4:50 PM
by asdev on 4/5/25, 4:29 PM