from Hacker News

Long Live the 'GPU Poor' – Open-Source AI Grants

by rajko_rad on 8/30/23, 4:37 PM with 110 comments

• by jph00 on 8/30/23, 8:59 PM

  I'm one of the recipients of the AI grants, to support my work at fast.ai. I'm extremely grateful to a16z for their support. Here's some additional details based on questions I see in the comments:

  My grant covered the purchase of a Scalar server from Lambda Labs, which allowed me to configure a system with 8 previous-gen A6000 GPUs, partly also thanks to NVIDIA who has recently given me 3 of those cards, and Lambda Labs who offered to provide everything at cost.

  a16z didn't ask for any equity or any kind of quid pro quo, other than to let folks know that they provided this grant. They couldn't have been more helpful through the process - I didn't have to fill out any forms (other than sign a single one page agreement), the contract was clear and totally fair (even explicitly saying that a16z wasn't going to receive any kind of rights to anything), and they wired me the money for buying the server promptly.

• by rajko_rad on 8/30/23, 5:22 PM

  Hi HN, we are super excited to announce this new initiative!!

  Please note, this program was designed to support individuals, teams, and hackers who are not pursuing commercial companies. Nevertheless, these projects push the state of the art in open source AI and help provide us with a more robust and comprehensive understanding of the technology as it is developed.

  We are really proud to be contributing in this small fashion and grateful to the first cohort and all others contributing in this space!

• by politelemon on 8/30/23, 5:11 PM

  Very happy to see oobabooga on the list. Everyone's contributing in different ways, and ooba serves as a fantastic portal to try out various models which I appreciate. Its readme even says it wants to be like the automatic 1111 of text generation. For beginners like me who are curious about this new world, ooba is a great entry point.

• by redox99 on 8/30/23, 5:41 PM

  I want to congratulate you based on your current grant recipients. They are all very impactful individuals/teams. It shows that you clearly did your research and put the money in the right places.

• by itissid on 8/30/23, 6:43 PM

  Noob Q: If I have 25000$ can I buy some GPUs and support the effort for supporting GPU Poor? Like is there a DIYer's guide to do this?

  I don't want to donate the cash but instead set up a handful of GPUs(Somehow?) and let people pay at cost to use them.

• by syntaxing on 8/30/23, 7:28 PM

  Whoa this is honestly awesome, a lot of tools I see on the list are ones I’ve been using so it’s great to see they’re getting financial support. Really hoping Ollama joins the list soon. I have some fun ideas I’ve been wanting to test out but probably requires a couple hours on 8X A100s + EvolveInstruct style data that cost upward of $1K+. Gonna start preparing some code to see if I can join grant #2!

• by tikkun on 8/30/23, 7:14 PM

  What are other open source LLM projects that people rely on? As a way to come up with ideas for future grant recipients.

  This is a good initiative, and an excellent initial batch. A great use of management fees!

  I'm struggling to think of additional ones because I feel like this covers everyone I would've suggested (particularly TheBloke, vLLM and oobabooga).

  Edit, some ideas for future grant recipients would be:

  * MLC LLM

  * ExLlama

  * An open source fork of text-generation-inference

  * AutoGPTQ

• by bodecker on 8/30/23, 6:22 PM

  Congrats to the folks involved.

  Even knowing this is partly motivated by branding/marketing, it's great to see a16z getting more aligned with solving real pain points (vs crypto and churning out shallow media in recent years). Hope they can keep it up and hopefully more "thought leaders"/VCs follow suit. Best of luck.

• by Havoc on 8/30/23, 5:30 PM

  Amazing. Those are really solid 1st picks. A huge chunk of the organic experimentation relies directly on them.

• by RcouF1uZ4gsC on 8/30/23, 6:47 PM

  > Eric Hartford: fine-tuning uncensored LLMs

  Makes me very happy that this is being undertaken.

  LLMs will be a very valuable tool. Having censorship built into these tools makes me very nervous. Imagine if Word refused to allow you to type any “bad word” in the name of safety and ethics.

• by mirekrusin on 8/30/23, 6:01 PM

  They should call this initiative Open AI.

• by htrp on 8/30/23, 6:03 PM

  ehartford and thebloke are definitely great picks advancing SOTA in underserved fields.

  Curious why there are no image ai projects receiving grants here?

• by minimaxir on 8/30/23, 5:31 PM

  It's interesting to note that there's particular emphasis on "removing censorship from LLM outputs" given a16z's investments outside the AI space in some ideological social media companies.

• by intalentive on 8/30/23, 5:43 PM

  TheBloke has had this sponsorship in his byline for a little while now.

• by zekrioca on 8/30/23, 6:56 PM

  I'm not sure why an application link wasn't provided. How is one suppose to reach them to apply?

• by seism on 8/30/23, 7:01 PM

  Hi Robin Hood. I wonder about the "continue their work without the pressure to generate financial returns" part. I don't see an expectation to work on non-commercial, or socially conscious, applications, just to commit to open source (what level of commitment also being unclear). It would actually be great if support programs along these lines were being offered by governments and public service institutions, with some serious governance and transparency around the process.

  Surely there are also some interesting econometrics on the "GPU poverty line" - someone has literature to share?

• by GaggiX on 8/30/23, 7:06 PM

  This reminds when a random chinese user finetuned the leaked NovelAI Stable Diffusion model and released it online, the Anything V3 model, tech company like Tencent started using it in their products.

• by UncleOxidant on 8/30/23, 5:45 PM

  "We believe artificial intelligence has the power to save the world"

  I don't agree with the side that says AI is about to destroy everything, it seems very hyperbolic... but neither do I agree with this sentiment that it's going to save the world.

• by ricardo81 on 8/30/23, 6:26 PM

  Leftfield thought, imagining a tax (vaguely like carbon credits) on energy/cpu cycles/modelling (just my vague idea). With the proceeds going to other projects to diversify the outcomes.

  Always seems to me our economy is being less based on money and more on energy and productivity, basically being rounded down to physics as a healthy market economy should.

• by wslh on 8/30/23, 5:40 PM

  We work in source code security auditing with semi-automated tools using AI and other techniques.

  Is this approach covered in the grants?

• by retrocryptid on 8/30/23, 6:12 PM

  I'm still not sure what I'm supposed to be using AI for. I already know how to code, tried for a decade to make ACR (auto content recognition) work with CNNs, and as best I can tell, chatGPT can write a news article for you if you write it first and then hire someone else to edit it.

• by stuaxo on 8/30/23, 7:44 PM

  This is great, but "dual 4090s" bring this stuff down to the Ryzen APU level.

• by jxf on 8/30/23, 5:56 PM

  Q: What are the grant amounts?

• by Michelangelo11 on 8/30/23, 5:28 PM

  This is great, but I'm not seeing anything about how to apply, or really anything about the application and selection process at all. Is this by design?

• by JonChesterfield on 8/31/23, 12:14 AM

  Is this a venture capital fund donating money to nvidia in order to support open source? It seems like some wires have got crossed along the way here

• by waihtis on 8/30/23, 5:35 PM

  hail e/acc and the thermodynamic god

• by artninja1988 on 8/30/23, 6:14 PM

  Thank you for doing this. Although I'm sure it's not purely for altruistic reasons, open source ai is important:)

• by satvikpendem on 8/30/23, 5:15 PM

  Well, at least this time a16z is giving money to actually useful technologies instead of bloviating about the world changing potential of cryptocurrencies. I imagine they've written down most of their cryptocurrency investments at this point.

  That being said, these are some great grants for local LLM builders, local computation for such powerful tools is needed, and will continue to be needed, as industry incumbents like OpenAI continue to not release their models.

• by svnt on 8/30/23, 5:39 PM

  “We are disguising this round of seed funding as a grant, accomplishing all of the below:

  1) brand-washing for people who don’t do the math

  2) avoiding angering our LPs and/or violating our fund theses and

  3) getting in on the ground floor of things that might follow OpenAI through the non-profit-to-VC-darling door

  Thanks for reading our press release, feel free to try to apply using your personal network (somehow) because we don’t care as we’ve already landed our positions.”

• by skilled on 8/30/23, 5:33 PM

  > We believe artificial intelligence has the power to save the world—and that a thriving open source ecosystem is essential to building this future.

  Thank you for understanding the fundamental flaws of the World and human consciousness, and thank you for spending multiple lifetimes as Buddhist monks (not to mention lamas) to understand the intricacies of the human mind from a scientific perspective. This is exactly what we need right now and this initiative looks to be the answer to all of our problems.

• by naillo on 8/30/23, 6:55 PM

  A16z is getting 'mad respect' from me for taking this line of the 'ai wars' and generally their attitude on all this. Really cool that they're going straight into 2010 era startup energy on this new wave. Idk I just find them really cool and inspirational.

• by thinktown on 8/30/23, 6:14 PM

  What was actually being offered here? Money? How much? This seems like such vague VC-marketing speak.

• by Kon-Peki on 8/30/23, 6:03 PM

  I'm glad you guys are sharing some money around, but I really have a hard time thinking of a person with "Dual 4090s" as "GPU Poor". LOL