by MasterScrat on 3/11/24, 3:23 PM
Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work.
I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!
by Flux159 on 3/11/24, 3:43 PM
This is nice to foster some competition in hardware for model training, but the availability of these machines seems very limited - I don't think there's any major cloud provider allowing per hour rental of Gaudi2 VMs and Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD. Availability and software stack is still heavily in Nvidia's favor right now, but maybe by the end of the year that will start changing.
by 1024core on 3/11/24, 4:29 PM
NVIDIA's profit margin is almost 92% on an H100. I'm surprised more chip companies haven't jumped on a "ML accelerator" bandwagon by now.
by ekelsen on 3/11/24, 3:32 PM
Some analysis of how and/or why it is able to be 3x faster despite no hardware metric being 3x better would make this actually useful and insightful instead of advertising.
by jsheard on 3/11/24, 3:26 PM
Hasn't H100 been shipping in volume for about a year already? Is Gaudi2 even available at comparable scale yet? I wouldn't count Nvidia out until they start slipping on similar timescales, i.e. if B100 doesn't have a clear lead over competing parts that become available at roughly the same time.
by ABS on 3/11/24, 3:44 PM
H100 was released almost exactly 1 year ago so I guess it's ok if Intel is now ready to compete with last year's model.
To those commenting about "no moat" remember CUDA is a huge part of it, it's actually HW+SW and both took a decade to mature, together
by yukIttEft on 3/11/24, 5:14 PM
I'm wondering how AI scientists work these days. Do they really hack Cudakernels or do they plug models together with highlevel toolkits like pytorch?
Considering its the latter, considering pytorch takes care of providing optimized backends for various hardwares, how big of a moat is Cuda then really?
by cherryteastain on 3/11/24, 5:19 PM
One question I have that nobody, including an Intel AXG employee, has been able to answer satisfsctorily for me is why both Gaudi and Ponte Vecchio exist. Wouldn't Intel have better chances of success if they focused on one product line?
by thunderbird120 on 3/11/24, 4:23 PM
Gaudi3 is supposedly due this year with a 4X bump in Bf16 training over Gaudi2. Gaudi is an interesting product. Intel seems to have something pretty decent but it hasn't seen much of a volume release yet. Maybe that comes with V3? Not sure exactly what their strategy with it is.
We do know that in 2025 it's supposed to be part of Intel's Falcon Shores HPC XPU. This essentially takes a whole bunch of HPC compute and sticks it all on the same silicon to maximize throughput and minimize latency. Thanks to their tile-based chip strategy they can have many different versions of the chip with different HPC focuses by swapping out different tiles. AI certainly seems to be a major one, but it will be interesting to see what products they come up with.
by tromp on 3/11/24, 3:24 PM
by BryanLegend on 3/11/24, 4:34 PM
This message was brought to you by Intel
by lostmsu on 3/13/24, 5:54 PM
I would potentially be interested in Gaudi-based workstation. Supermicro servers seem good, but they do not have DisplayPort outputs, and jury-rigging them on is not something I'd do.
by mittermayr on 3/11/24, 3:28 PM
Frankly, this may be good to level out the market a bit. While it's been fun to see Nvidia rise up through this insanity, it would only be healthy to have others catch up here and there eventually.
by qeternity on 3/11/24, 5:54 PM
Has anyone been running LLMs on TPUs in prod? Curious to hear experiences.
by mistrial9 on 3/11/24, 5:34 PM
https://es.wikipedia.org/wiki/Antoni_Gaud%C3%ADGaudi is a famous name for a reason.. the flowing lines and frankly, nonsense and silliness, in the art and architecture of Gaudi stands for generations as a contrast to the relentless severity of formal classical arts (and especially a contrast to Intel electronic parts).
by CrocODil on 3/12/24, 9:59 AM
Does the performance picture change with Int8?
by throwaway4good on 3/11/24, 5:45 PM
Who fabs the Gaudi2? TSMC or Intel themselves?
by memossy on 3/11/24, 2:56 PM
"For Stable Diffusion 3, we measured the training throughput for the 2B Multimodal Diffusion Transformer (MMDiT) architecture model. Gaudi 2 trained images 1.5x faster than the H100-80GB, and 3x faster than A100-80GB GPU’s when scaled up to 32 nodes. "