from Hacker News

ChatGPT Is Expensive to Run

by ovaistariq on 4/24/23, 5:28 PM with 4 comments

• by brucethemoose2 on 4/24/23, 5:39 PM

  Yeah, and its a waste. Nvidia runs the A100s in a relatively inefficient power band (300W or 400W), and tons of power is burned on the interconnect huge LLMs need to fit in memory.

  And the servers cost a fortune, with a huge profit margin.

  Its not sustainable... in fact, its probably less efficient than the crypto mining boom, as miners were downclocking GPUs (and building simple ASICs) to run at more efficient voltages.

• by skilled on 4/24/23, 5:47 PM

  Recent discussion (3 days ago): https://news.ycombinator.com/item?id=35652434