from Hacker News

Contemplative LLMs

by zora_goron on 1/12/25, 12:02 AM with 10 comments

• by lsy on 1/12/25, 5:15 AM

  The title makes it sound like some new architecture, but this is a blog post where someone likes the results they get sometimes when they fiddle with their input to the LLM to suggest “contemplation”, which apparently makes the LLM generate a large paragraph of highly neurotic text before the answer. There aren’t benchmarks or investigation of the model to see whether it is robust or generalizable so it’s hard to say whether this is useful or not.

• by padolsey on 1/12/25, 5:18 AM

  Cool! I love seeing the very subtle emergent behaviours of different CoT approaches. I reckon people still don't fully appreciate the brittle artistic subtlety of trying to make something akin to logic emerge from these weird transformer machines.

  > Now, instead asking the model to respond in JSON we use XML tags to separate the start and end of the contemplation phase and the final answer

  I suspect the author wisely avoided function-calling/JSON since it doesn't guarantee sequence. This–and a few other frailties–make me almost always use XML-like markup for my LLM API calls.

  Markup langs like XML and HTML lend themselves quite beautifully to this task. They are stream-friendly, semantically enriched, leniently parseable (html was designed in part for fallible humans to write and for browsers to incrementally render) and by nature of being "markup" they are complementary to the autoregressive nature of LLMs. One assumes as well that tonnes of prose appears in HTML found in training corpuses, less so in JSON which is usually used for transactional data and RPC-like things, which must surely bias JSON completions to more robotic formations. FWIW I ended up creating a library (github.com/padolsey/xmllm) to help me get structured data from LLMs using XML (through the forgiving eyes of an HTML parser), so that I never have to rely on specific LLM tool/function-calling abstractions. Even tiny models like Qwen2.5 and Ministral3B have pretty superb (x|ht)ml compliance, much less so with JSON.

• by vunderba on 1/12/25, 5:01 AM

  Isn't this just a much larger "prompt based equivalent" of chain-of-reasoning systems like Qwq?

  https://qwenlm.github.io/blog/qwq-32b-preview

• by m3kw9 on 1/12/25, 4:56 AM

  If they were confident of the answer at first even when using contemplating as the context, why would they start saying “that doesn’t seem right”? And then re work the answer

• by riwsky on 1/12/25, 5:20 AM

  Pierre Menard, Author of the “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” Paper