from Hacker News

Ask HN: What does your AI tech stack look like?

by thepaulthomson on 7/14/23, 2:15 AM with 14 comments

We recently did a review of our AI tech stack at (both internal and external).

Some of our favorites and more prolifically used were OpenAI API, ChatGPT, Midjourney, and Jasper.

Curious to hear what other AI tools (and frameworks like LangChain etc) y'all are heavily dependent on

  • by brucethemoose2 on 7/14/23, 2:33 AM

    I am all local

    VoltaML for stable diffusion imagegen (as AITemplate is really fast, and controlnet and LORAs are super useful)

    Llama.cpp for textgen... I use the koboldcpp frontend (and sometimed AI Horde) for experimenting with models, though I am still figuring out the exact stack I want for productivity. I am watching this PR closely:

    https://github.com/ggerganov/llama.cpp/pull/1773

    As well as Starcoder/Openllama finetunes for summarization, codegen and such.

  • by ouraf on 7/14/23, 2:43 PM

    OpenAI + PineconeDB -> Langchain -> Quivr to get a decent UI

    right now i'm trying to prove to the company i work that you can turn the endless rules and daily information stream into something people can find easily using sematic search (vector DB) and an AI to summarize categorize, generate embeddings and auto update the DB, while also interpreting the content of the search result and giving a more digestible answer and a source link if the end user needs more information.

    Keyword based search is a pain when different products have similar names or the internal search doesn't filter words like "of" and "and" from the keyword search.

    After that, if the tests work and i can push the idea forward, it'll probably be OpenAI + a local vector DB (Chroma?) + custom made search page for internal usage

  • by saqadri on 7/16/23, 4:31 PM

    For us the complexity has all been about the infra around the AI stack. For example, having a k8s cluster for running inference for open source models (Stable Diffusion, Bark, etc.). Beyond that it’s been custom abstractions around Open AI/PaLM APIs and vector stores (pinecone and Faiss).

    This stack is evolving so quickly but we are convinced it goes back to a distributed systems/infra problems when it comes to complexity, unless you’re training your own foundation models.

  • by thepaulthomson on 7/14/23, 2:16 AM

    A deeper look into our stack if you're interested: https://www.commandbar.com/blog/harnessing-ai-magic

  • by quickthrower2 on 7/14/23, 9:56 AM

    Right now I am playing with modal.com, trying to run some pytorch code lol. Not sure if that counts as I am just learning.