from Hacker News

Meta MTIA v2 – Meta Training and Inference Accelerator

by _yo2u on 4/10/24, 3:16 PM with 60 comments

  • by jsheard on 4/10/24, 4:27 PM

    I like the interactive 3D widget showing off the chip. Yep, that sure is a metal rectangle.

  • by modeless on 4/10/24, 3:47 PM

    Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.

  • by mlsu on 4/10/24, 6:52 PM

    Certainly an interesting looking chip. It looks like it's for recommendation workloads. Are those workloads very specific, or is there a possibility to run more general inference (image, language, etc) on this accelerator?

    And, they mention a compiler in PyTorch, is that open sourced? I really liked the Google Coral chips -- they are perfect little chips for running image recognition and bounding box tasks. But since the compiler is closed source it's impossible to extend them for anything else beyond what Google had in mind for them when they came out in 2018, and they are completely tied to Tensorflow, with a very risky software support story going forward (it's a google product after all).

    Is it the same story for this chip?

  • by chessgecko on 4/10/24, 5:49 PM

    I thought MTIA v2 would use the mx formats https://arxiv.org/pdf/2302.08007.pdf, guess they were too far along in the process to get it in this time.

    Still this looks like it would make for an amazing prosumer home ai setup. Could probably fit 12 accelerators on a wall outlet with change for a cpu, would have enough memory to serve a 2T model at 4bit and reasonable dense performance for small training runs and image stuff. Potentially not costing too much to make either without having to pay for cowos or hbm.

    I'd definitely buy one if they ever decided to sell it and could keep the price under like $800/accelerator.

  • by teaearlgraycold on 4/10/24, 4:39 PM

    Still seems pretty primitive. Very cool though.

    I can only imagine the lack of fear Jensen experiences when reading this.

  • by prng2021 on 4/10/24, 11:00 PM

    3x performance but >3x TDP. Am I missing something or is that unimpressive?

  • by jrgd on 4/10/24, 5:13 PM

    I find it weird that not everyone agree Meta and Facebook and social networks in general are doing some good the the society and our democracies; yet they manage to spend incredible amount of money/energy/time to develop solutions to problems we aren't exactly sure are worth solving…

  • by duchenne on 4/10/24, 6:23 PM

    Is it possible to buy it?

  • by ein0p on 4/10/24, 7:43 PM

    Come on, Zuck, undermine Google Cloud and take NVIDIA down a few pegs by offering this for purchase in good quantities.

  • by sroussey on 4/10/24, 3:57 PM

    Pretty large increase in performance over v1, particularly in sparse workloads.

    Low power 25W

    Could use higher bandwidth memory if their workloads were more than recommendation engines.

  • by throwaway48476 on 4/10/24, 6:37 PM

    It's interesting that they are not separating training and inference.

  • by xnx on 4/10/24, 4:00 PM

    My mind still boggles that a BBS+ads company would think it needs to design its own chips.

  • by bevekspldnw on 4/10/24, 10:07 PM

    Pretty fascinating they mention applications for ad serving but not Metaverse.

    I feel like Zuck figured out he’s just running an ads network, the world is a long way anway from some VR fever dream, and to focus on milking each DAU for as many clicks as possible.