from Hacker News

Ask HN: Can I download GPT / ChatGPT to my desktop?

by AHappyCamper on 12/29/22, 3:44 PM with 48 comments

• by 5e92cb50239222b on 12/29/22, 3:57 PM

  OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use:

  https://en.wikipedia.org/wiki/GPT-3

• by turkeygizzard on 12/29/22, 4:04 PM

  I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home.

  Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able to manage context/understanding beyond this window length

• by htns on 12/29/22, 4:27 PM

  I don't much follow AI news beyond what I randomly happen to see on HN, but this might still be the largest open source model: https://github.com/yandex/YaLM-100B . There's discussion of it here: https://old.reddit.com/r/MachineLearning/comments/vpn0r1/d_h... - at the bottom of that page is a comment from someone who actually ran it in the cloud.

• by Sharlin on 12/29/22, 4:16 PM

  Even if it were freely available, there's no way to run GPT3 or ChatGPT on any existing desktop hardware. The exact hardware requirements aren't public either (yes, very "open") but a full 175-billion-parameter GPT3 instance requires hundreds of gigabytes of GPU memory, and even though ChatGPT is "smaller and better", when it comes to conversational dialogue, there's no way to fit it in current consumer hardware.

• by navjack27 on 12/29/22, 4:53 PM

  You can do gpt-j

  https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4...

  I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2

• by fred967 on 12/29/22, 7:26 PM

  You can download his ancestor here:

  https://winworldpc.com/product/dr-sbaitso/2x

• by mellosouls on 12/29/22, 4:35 PM

  It's not possible post GPT2 for the reasons given by others.

  Open communities with potential for involving yourself include Hugging Face and EleutherAI, the former perhaps more accessible, the latter an active Discord.

  It's been a while since I spent time looking at them, I'm not sure if there is something you can easily get up and running with.

  https://huggingface.co/

  https://www.eleuther.ai/

• by dragonwriter on 12/29/22, 5:27 PM

  There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”.

  You probably won't be able to run (or especially train) them on typical desktops, though.

  [0] https://www.eleuther.ai/projects/gpt-neox/

• by chamwislothe2nd on 12/29/22, 7:24 PM

  https://github.com/bigscience-workshop/petals

  Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!

• by trilbyglens on 12/29/22, 10:01 PM

  My feeling is that even if it were available to download, the compute requirements to run it at production speeds would likely be blistering.

• by PlotCitizen on 12/29/22, 3:56 PM

  It’s not possible currently but there’s another story on the HN front page with an open source alternative which I haven’t tried

• by luckyme123 on 12/29/22, 4:11 PM

  Luckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)