from Hacker News

The surprising effectiveness of test-time training for abstract reasoning [pdf]

by trott on 11/11/24, 4:23 PM with 27 comments

• by mikeknoop on 11/11/24, 5:54 PM

  Context: ARC Prize 2024 just wrapped up yesterday. ARC Prize's goal is to be a north star towards AGI. The two major categories of this year's progress seem to fall into "program synthesis" and "test-time fine tuning". Both of these techniques are adopted by DeepMind's impressive AlphaProof system [1]. And I'm personally excited to finally see actual code implementation of these ideas [2]!

  We still have a long way to go for the grand prize -- we'll be back next year. Also got some new stuff in the works for 2025.

  Watch for the official ARC Prize 2024 paper coming Dec 6. We're going to be overviewing all the new AI reasoning code and approaches open sourced via the competition [3].

  [1] https://deepmind.google/discover/blog/ai-solves-imo-problems...

  [2] https://github.com/ekinakyurek/marc

  [3] https://x.com/arcprize

• by arjvik on 11/11/24, 6:34 PM

  Test-Time Training is incredibly powerful. Most recently, it has been shown that Self-Attention can in fact be viewed through the lens of test-time training, with a kernel-smoother "learning" from context. Simply replacing that with more powerful models than a kernel-smoother result in very capable and scalable models!

  https://arxiv.org/abs/2407.04620

• by sthlmb on 11/11/24, 6:15 PM

  I initially read that as "Tea-Time" training and my inner Brit got a little excited..

• by zbyforgotp on 11/11/24, 6:38 PM

  Is test time the same thing as inference time?