from Hacker News

In-browser LLM inference engine with WebGPU and OpenAI API

by CharlieRuan on 6/13/24, 7:27 PM with 4 comments

• by crowwork on 6/13/24, 7:30 PM

  Comes with ability to do full structured generation with json schema

  also a in-browser demo https://chat.webllm.ai/

• by leshenj15 on 6/13/24, 10:30 PM

  So cool. No more privacy concerns!

• by ruihangl on 6/13/24, 7:44 PM

  It's good to see that it contains the standard OpenAI API interface in JavaScript which looks very convenient.

• by cyx6 on 6/13/24, 10:25 PM

  AMAZING!