from Hacker News

BigCode Project Releases StarCoder: A 15B Code LLM

by enum on 5/4/23, 5:42 PM with 37 comments

• by enum on 5/4/23, 5:42 PM

  StarCoder is a 15B LLM for code with 8k context and trained only on permissive data in 80+ programming languages.

  Release thread: https://twitter.com/BigCodeProject/status/165417494197606811...

  Model: https://huggingface.co/bigcode/starcoder

  Paper: https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0k...

  Also available in HuggingChat: https://huggingface.co/chat

• by captn3m0 on 5/4/23, 6:55 PM

  License text: https://huggingface.co/spaces/bigcode/bigcode-model-license-.... Attachment A at the very end has the usage restrictions.

• by djoldman on 5/4/23, 6:42 PM

  Paper: https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0k...

  Evaluation results are impressive!

• by anentropic on 5/4/23, 6:59 PM

  https://huggingface.co/spaces/bigcode/bigcode-playground

  Can someone explain to me the "Model Formats" section at the end of that page. It makes sense to me as a description of how to format the training data, but it also says "Use these templates to explore the model’s capacities"

  How would you make use of this prefix format in a prompt?

    <reponame>REPONAME<filename>FILENAME<gh_stars>STARS
    code<|endoftext|>

• by kannangce on 5/5/23, 7:36 PM

  Is it possible to load the model for a particular language. Thus minimising the required resources.

• by anentropic on 5/4/23, 6:50 PM

  The org page has a link to https://huggingface.co/bigcode/starencoder "Encoder model trained on TheStack." which is currently a 404

  Just curious what that is vs the main https://huggingface.co/bigcode/starcoder model

• by kateklink on 5/4/23, 7:18 PM

  I'm excited to see all the model's implementations that are yet to come. Refact.ai seems to be working on code transformation tools and chat inside Jetbrains and VSCode powered by StarCoder https://refact.ai/blog/2023/self-hosted-15b-code-model/

• by eclark on 5/4/23, 7:39 PM

  The training cluster is super impressive, any ideas on how to fine tune StarBase for a smaller language than python on something more attainable?

• by SomeHacker44 on 5/4/23, 10:32 PM

  i was not able to find the 80 programing languages it supports using my phone. you would think this would be front and center somewhere.

• by tmaly on 5/4/23, 9:30 PM

  I might have missed it, what are the minimum hardware requirements needed to run this?

• by syntaxing on 5/4/23, 7:49 PM

  So many things are 404 on this announcement. The demo looks really amazing though, I’m pretty excited to try this out locally.