from Hacker News

WebGPU GPT Model Demo

by b_mc2 on 4/21/23, 12:54 AM with 29 comments

Tweet by creator: https://twitter.com/willdepue/status/1649147091573432321

  • by DustinBrett on 4/21/23, 2:26 AM

    It indeed works and loads quick. I am more interested currently in the Vicuna 7B example from https://mlc.ai/web-llm/

    Also instead of just "Update Chrome to v113" the domain owner could sign up for an origin trial https://developer.chrome.com/origintrials/#/view_trial/11821...

  • by FL33TW00D on 4/21/23, 7:29 AM

    My 250M parameter model runs in 50ms/token ;)

    Releasing April 26th when Chrome 113 hits stable. Open source NPM library you can add to any project.

    Preview here: https://twitter.com/fleetwood___/status/1646608499126816799?...

  • by doodlesdev on 4/21/23, 2:08 AM

       > WebGPU is supported in your browser!

   > Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds.
    Anyone using Chromium care to chime in?

    If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.

  • by ianpurton on 4/21/23, 9:04 AM

    Question. I can see in the code the WGSL that's needed to implement inference on the GPU. https://github.com/0hq/WebGPT/blob/main/kernels.js

    Could this code also be used to train models or only for inference?

    What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?

  • by MuffinFlavored on 4/21/23, 3:35 AM

    > At the moment, WebGPT averages ~300ms per token on GPT-2 124M running on a 2020 M1 Mac with Chrome Canary.

    How do ChatGPT on GPT-3.5 / GPT-4 compare?

  • by luizfelberti on 4/21/23, 3:20 PM

    It's really a shame that there is no 8-bit float support in the WebGPU spec. Even though few cards support it natively, it'd still massively benefit ML workloads.

    Another annoying constraint but specific to wgpu (Rust's implementation of WebGPU) is that it does not support f16 yet (which IS in the spec), only through SPIR-V passthrough...

  • by tormeh on 4/21/23, 10:31 AM

    Any way to run this kind of thing outside the browser? Chrome hasn't enabled WebGPU on Linux yet.

  • by kristianp on 4/22/23, 1:01 AM

  • by junrushao1994 on 4/21/23, 4:03 AM

    Is there any plan to support larger models than GPT-2?

  • by eurekin on 4/21/23, 1:57 PM

    Omg, no pytorch/wsl/conda hiccups... This could save me some sleepless nights

  • by samueldurante on 4/22/23, 5:52 AM

    I didn't understand why I need WebGPU to use WebGPT...