from Hacker News

Trillium TPU Is GA

by gok on 12/11/24, 3:44 PM with 80 comments

• by xnx on 12/11/24, 5:40 PM

  > We used Trillium TPUs to train the new Gemini 2.0,

  Wow. I knew custom Google silicon was used for inference, but I didn't realize it was used for training too. Does this mean Google is free of dependence on Nvidia GPUs? That would be a huge advantage over AI competitors.

• by lanthissa on 12/12/24, 12:59 PM

  Okay I really dont understand this, nvidia has a 3.4T market cap google has a 2.4T post run up, and its PE is like 38 vs 25 so its a higher multiple on the business too. It appears making the best AI chip is a better business than googles entire conglomerate.

  If TPU's are really that good why on earth would google not sell them. People say its better to rent, but how can that be true when you look at the value of nvidia.

• by blackeyeblitzar on 12/11/24, 7:55 PM

  So Google has Trillium, Amazon has Trainium, Apple is working on a custom chip with Broadcom, etc. Nvidia’s moat doesn’t seem that big.

  Plus big tech companies have the data and customers and will probably be the only surviving big AI training companies. I doubt startups can survive this game - they can’t afford the chips, can’t build their own, don’t have existing products to leech data off of, and don’t have control over distribution channels like OS or app stores

• by randomcatuser on 12/11/24, 5:57 PM

  How good is Trillium/TPU compared to Nvidia? It seems the stats are: tpu v6e achieves 900 TFLOPS per chip (fp16) while Nvidia H100 achieves 1800 TFLOPS per gpu? (fp16)?

  Would be neat if anyone has benchmarks!!

• by WanderPanda on 12/12/24, 3:48 AM

  Crazy conglomerate discount on Alphabet if you can see TPUs as the only Nvidia competitor for training. Breaking up Alphabet seems more profitable than ever

• by teleforce on 12/13/24, 3:22 AM

  It's beyond me why processor with dataflow architecture is not being used for ML/AI workloads, not even in minority [1]. Native dataflow processor will hands down beats Von Neumann based architecture in term of performance and efficiency for ML/AI workloads, and GPU will be left redundant for graphics processing instead of being the default co-processor or accelerator for ML/AI [2].

  [1] Dataflow architecture:

  https://en.wikipedia.org/wiki/Dataflow_architecture

  [2] The GPU is not always faster:

  https://news.ycombinator.com/item?id=42388009

• by LittleTimothy on 12/12/24, 1:18 PM

  The question I'd like to know the answer to is "What was the total cost of training Gemini 2.0 and how does it compare to the total cost to train equivalent capability models on Nvidia GPUs?". I'd be fascinated to know, and there must be someone at Google who has the data to actually answer that question. I suspect it's politically savvy for everyone at Google to pretend that question doesn't exist or can't be answered (because it would be an existential threat to the huge TPU project), but it would be absolutely fascinating. In the same way that Amazon eventually had to answer the "Soo.... how much money is this Alexa division actually making" question.

• by amelius on 12/11/24, 7:34 PM

  Are the Gemini models open?

• by Hilift on 12/11/24, 8:40 PM

  "we constantly strive to enhance the performance and efficiency of our Mamba and Jamba language models."

  ... "The growing importance of multi-step reasoning at inference time necessitates accelerators that can efficiently handle the increased computational demands."

  Unlike others, my main concern with AI is any savings we got from converting petroleum generating plants to wind/solar, it was blasted away by AI power consumption months or even years ago. Maybe Microsoft is on to something with the TMI revival.