from Hacker News

SGLang: Fast and Expressive LLM Inference with RadixAttention for 5x Throughput

by covi on 2/21/24, 4:56 PM with 0 comments