from Hacker News

Amazon has a way to scrape GitHub and feed its AI model

by doubtfuluser on 6/15/24, 10:34 AM with 59 comments

• by Kye on 6/15/24, 11:28 AM

  Is it git pull?

  >> "In response, Amazon proposed a workaround: encouraging its employees to create multiple GitHub accounts and share their access credentials."

  Ah, no, it's git pool.

• by xmodem on 6/15/24, 11:54 AM

  Ethically Microsoft has about as much claim to be able to use the data for co-pilot as anyone else.

  On the other hand, maybe a MSFT v Amazon lawsuit over this could be the wake up call the world needs that maybe we should stop centralising critical infrastructure in the hands of a single company. Which is why I think they wouldn't do it - at most I could see Microsoft tightening request limits on accounts associated with Amazon.

• by jsnell on 6/15/24, 12:05 PM

  I'm surprised Amazon's legal team signed off on this. It's clearly against the GitHub terms of service[0], and Amazon employees acting on the instructions from Amazon had to approve those terms. It seems pretty much identical to the LinkedIn vs. hiQ scraping case, where as I understand the fake account creation was the key point.

  [0] E.g. no API key sharing for the purposes of evading rate limits, only a single free account per person or organization.

• by koolba on 6/15/24, 12:02 PM

  Is the cover image itself generated via some ML model? The old guy in the middle is missing substantial parts of his arm. The box right by him also has some artifacting in the corner.

• by lokimedes on 6/15/24, 12:52 PM

  This just rekindled my desire to self-host my git repos. The whole idea that a platform provider can use the IP I host there is obscene. That thieves steal by bounty from each other is not the story.

• by neilv on 6/15/24, 12:27 PM

  Separate from the courts, Microsoft could send a message to the AI gold rush field, about "abuse of Microsoft's resources", via ToS:

  * All Amazon domain names could be banned from accounts on GitHub, or face annoying restrictions, implemented with trivial technical changes. And lawyers could send a letter to Amazon legal, about how Amazon may and may not use GitHub, including Amazon personnel having to disclose their affiliation (not hide it with GMail), and craft some language about how those employee accounts may and may not be used.

  * More harshly, but fear-instilling to individuals throughout industry, the individuals who let their accounts be used for the scraping could be banned from GitHub, for ToS violation. Not only those particular accounts, but any accounts the individuals might use. (This would hurt, not only for genuine open source participation, but also given how open source is sometimes used for job-hunting appearances, and all the current employers that ask for candidate's "GitHub" specifically rather than open source in general.) If banning would have undesired effects of projects GitHub wants to host being pulled, or public reaction as too harsh and questioning why GitHub has so much power, there could instead be annoying restrictions.

• by foreigner on 6/15/24, 12:53 PM

  Microsoft could sabotage Amazon's AI model by returning poisoned code to accounts registered with @amazon.com email addresses.

• by raarts on 6/15/24, 12:13 PM

  Language in this article smells like it's written or rewritten by AI.

• by paradite on 6/15/24, 12:20 PM

  Microsoft is probably one of the few companies that can sue Amazon without worrying about retaliation from Amazon.

  For example, GitLab would need to think twice before suing because they offer deployment on AWS.

• by threecheese on 6/15/24, 1:27 PM

  Can anyone share a Fermi estimation of the size of poison-pill training data required to impact code interpreter models? (of the size that AMZN might be building with this data)

  I expect it would vary by language/platform popularity (size of available training code). Is it infeasible to create or generate enough code, pushed to enough repositories, to impact the correctness of a model that includes the code in its training data set?

• by lofaszvanitt on 6/15/24, 2:01 PM

  MS only provides the infra, everything else is other's hard work under the trojan horse open source whatever. If they introduce limits, time to leave github. This will evolve into an elsevier vs researchers kinda situation.

• by chumanak on 6/15/24, 3:33 PM

  This article doesn’t make any sense. Why would Amazon make their employees do all this when they can easily pay for a service like crawlbase or similar and easily scrape github without having to create employee accounts?

• by rty32 on 6/15/24, 12:17 PM

  If github cares enough about this, they would have already sued Amazon. I don't think the author needs to worry about any of this

• by hi-v-rocknroll on 6/15/24, 11:27 AM

  MSFT's LinkedIn scraping was also a thing about 10 years ago until the magic method was taken away. :'(

• by amadeuspagel on 6/15/24, 12:32 PM

  They should send make this data available for everyone on AWS.

• by glimshe on 6/15/24, 11:28 AM

  I couldn't care less about these huge tech companies stealing from one another. Let them sue themselves to extinction.

• by htrp on 6/15/24, 11:59 AM

  disappointing that large mega Corp does the exact same thing broke developers do to get around rate limits