from Hacker News

Longwriter – Increase llama3.1 output to 10k words

by taikon on 10/7/24, 2:05 PM with 29 comments

• by vessenes on 10/7/24, 4:45 PM

  The sample output is interesting - it has highly suggestive chapter titles which read like pretty normal story beats. It seems like it's guiding itself on these, then able to chunk out longer form writing per chapter.

  For what it's worth, the writing is .. bland. In the way that only an LLMs writing can be -- relatively grammatically sound, and totally soulless. I will never think of the love story of Elizabeth and Thomas again, despite having read the entire thing.

  In early days of GPT-3, I experimented a lot with getting it respond as certain authors, and it was really quite excellent at that. This is one of the many things that seem likely to have been nerfed over time, I'd guess partly because human preference training just asks for bland responses, and partly because the injected prompts from OpenAI strongly discourage doing things related to real people, and those preferences are carried through, subtlely or not, into the augmented training data most open models tune on.

• by mmaunder on 10/7/24, 5:11 PM

  What the difference between this and using chat history to concatenate outputs and prompting with something like “Now write the next section” repeatedly? I’ve done that with NotebookLM and it’ll write a complete fictional story based on sources, for example.

• by ed on 10/7/24, 4:17 PM

  Paper: https://arxiv.org/abs/2408.07055

  The model is stock llama, fine tuned with a set of long documents to encourage longer outputs.

  Most of the action seems to happen in an agent.

• by danng87 on 10/7/24, 3:35 PM

  Interesting project!

  Does anyone know how LongWriter handles maintaining coherence and structure in longer outputs? Also, are there specific strategies or parameters recommended for fine-tuning LLaMA 3.1 with this setup to maximize the quality of generated text?

• by 8bitsrule on 10/8/24, 1:22 AM

  Obviously there needs to be some oversight that limits distribution of machine-written articles 'created' by people that aren't already competent in the subject-area to know whether the content is trustworthy. Otherwise a lot of damage can be done in a very short time. (Pandora's box and all that.)

• by hshshshsvsv on 10/8/24, 2:22 PM

  Will this ever be able to generate anything meaningful or useful?

• by wkat4242 on 10/7/24, 5:46 PM

  Really interesting. I wonder if you can do this on ollama too.

• by jw12 on 10/8/24, 1:09 PM

  Looks super interesting!

• by fitsumbelay on 10/8/24, 12:30 AM

  this looks cool. any plans to support 3.2?

• by alwinaugustin on 10/7/24, 3:26 PM

  How to use this with the local ollma setup