from Hacker News

Meta to release open-source commercial AI model

by maskil on 7/14/23, 2:45 PM with 159 comments

  • by foob on 7/14/23, 4:14 PM

    From the recent story about the Sarah Silverman lawsuit:

    The complaint lays out in steps why the plaintiffs believe the datasets have illicit origins — in a Meta paper detailing LLaMA, the company points to sources for its training datasets, one of which is called ThePile, which was assembled by a company called EleutherAI. ThePile, the complaint points out, was described in an EleutherAI paper as being put together from “a copy of the contents of the Bibliotik private tracker.” Bibliotik and the other “shadow libraries” listed, says the lawsuit, are “flagrantly illegal.”

    IANAL, but this basically sounds like LLaMa was trained on illegally obtained books by Meta's own admission. It's an exciting development that Meta is releasing a commercial-use version of the model, but I wonder if this is going to cause issues down the road. It's not like Meta can remove these books from the training set without retraining from scratch (or at least the last checkpoint before they were used).

    [1] https://news.ycombinator.com/item?id=36657540

  • by zargon on 7/14/23, 3:28 PM

    It's not open source, it's freeware or something like that. Weights aren't the source code of LLMs, they're the binaries.

  • by greatpostman on 7/14/23, 3:32 PM

    Meta is going to ruin open ais moat on purpose. Great business strategy and good for everyone but metas competitors

  • by ekojs on 7/14/23, 3:23 PM

    Seems that the source is a FT article that was discussed yesterday: https://news.ycombinator.com/item?id=36712168

    From the FT article: '“The goal is to diminish the current dominance of OpenAI,” said one person with knowledge of high-level strategy at Meta.'

  • by forgingahead on 7/14/23, 4:36 PM

    Zuck is a total killer. What better way to fight Google and Microsoft than to effectively spawn thousands of potential competitors to their AI businesses with this (and other) releases. There will be a mad scramble over the released weights to develop new tech, these startups will raise tons of money, and then fight the larger incumbents.

    This is not charity, this is a shrewd business move.

  • by whimsicalism on 7/14/23, 5:03 PM

    If you read past the title, this article is not at all clear if they are referring to a commercial offering (ie. license our model for $$) or an open-source license with commercial usage (Apache, etc.)

    My guess is still the latter because that's what I've heard the rumors about, but this article is pretty unclear on this fact.

  • by pmarreck on 7/14/23, 4:38 PM

    I have a 128 core Threadripper, a 2080 Ti and a 3080 Ti.

    How can I play with open source LLM's locally?

  • by loufe on 7/14/23, 6:11 PM

    I'm surprised nobody here has brought up the sensorship in this model. Listening to Mark Zuckerberg on Lex Friedman's podcast talk about it, it sounds like the model will be significantly blunted vs its "research" version release.

  • by stale2002 on 7/14/23, 4:37 PM

    I remember arguing with people who honest to god thought that LLAMA was some sort of secret ploy, to trick startups into using it, so that meta could sue them for using it commercially.

    Well now there is a commerical release. I guess it wasn't some corporate plot after all!

    Some people just can't admit when a corporation does a good thing.

    (In this case, the good thing is being done to obsolete their competitors, but it is good none the less, that a commerical LLM is available for people to use for free)

  • by obblekk on 7/14/23, 3:48 PM

    Maybe they've solved the fingerprinting problem and can identify text generated from their model, and this is a way of discovering the market they can sell more advanced models to directly. B2B leadgen...

  • by 0cf8612b2e1e on 7/14/23, 4:17 PM

    From my quick skim I could not find a date. Any idea when this might happen?

  • by rvz on 7/14/23, 4:58 PM

    See. They don't care about the LLaMA model leak. It turns out that it was OpenAI that cares because it ruins their moat. It costs Meta nothing to release a better open-source or freely available version of LLaMA again.

    Still waiting for the 'Meta is dying' and 'Fire Mark Zuckerberg' calls from last year. A year later, where are they now?

  • by TheBengaluruGuy on 7/15/23, 2:11 AM

    This conversation triggers a thought.

    Does it mean that any blogs that I wrote from my own insights, will automatically be trained on the model… without my permission?

    As an author, it feels like it’s stealing the knowledge and insight without appropriate attribution.

  • by Jeff_Brown on 7/14/23, 3:13 PM

    What's the monetization model here? Is this a closed-source version of their open-source model? (That's suggested by the phrase in the article, "a commercial version of LLaMA, its open-source large language model".)

  • by sagebird on 7/14/23, 6:16 PM

    repeat after me:

    hardware is the only moat

    If you want to live the good life before you are exquisitely extinguished, spend every other day figuring out how to buy more NVDA, the other days exercising outside, being human.

  • by bilsbie on 7/14/23, 6:49 PM

    Is it possible to do further training on the weights they release?

  • by 40yearoldman on 7/14/23, 3:28 PM

    Is the title an oxymoron?

    Open-source commercial?