from Hacker News

Training NanoGPT on My Journal

by romseb on 1/29/23, 8:02 PM with 24 comments

  • by downrightmike on 1/30/23, 2:35 AM

    "I have been the happiest when She doesn't want to break up" wow that is saddening, you have to shoot higher dude, you deserve it!

  • by svnt on 1/29/23, 9:51 PM

    It sounds like your incantations may have resurrected Racter.

    https://en.m.wikipedia.org/wiki/Racter

  • by sigmoid10 on 1/30/23, 11:27 AM

    >The output is not great, but at least there are some sentences that make sense :) I will do some more neural network training and dataset cleaning, let's see where it will get me.

    From my own experience doing something very similar, I'm afraid that this won't cut it. Even a 1B parameter language model will struggle with the most simple topics. The smallest thing that I have seen, that may be useful for general purpose, is GPT-J with 6B parameters and even that is far from perfect. The idea of running these architectures in any useful way on old hardware is sadly a pipe dream. AI got where it is today mostly because of the increase in computing power.

  • by rgbrgb on 1/30/23, 5:22 AM

    > All in all, this AI-generated line describes my feelings about the results perfectly:

    > You're a real problem, but I'm just happy that you are doing.

    Good line.

  • by sebastianconcpt on 1/30/23, 1:02 AM

    This is something that I've imagined playing with eventually.

    Are you thinking in doing it able to "chat"? (integrate your new input)

  • by alsargent on 2/3/23, 2:47 PM

    I wonder if note taking applications like Notion (or new startups) might include something this as a capability to improve their search and summarization.

  • by antonvs on 1/30/23, 11:00 AM

    > my laptop has less than 1GB

    What gives? Most smartphones made in the last 10 years have more RAM than that.

  • by Romen_b on 1/30/23, 7:53 AM

    Pretty interesting use of the LLMs.

  • by fuddle on 1/30/23, 4:48 AM

    I'd be interested to know how much text it was trained on.

  • by smarterbot on 1/30/23, 7:17 AM

    Cool, running a chat bot on your laptop.