from Hacker News

AIConfig – source control format for gen AI prompts, models and settings

by saqadri on 11/17/23, 5:06 PM with 16 comments

  • by saqadri on 11/17/23, 6:00 PM

    Hi HN! OP here - this is our first open source project and we are really looking for your feedback about this direction, because we haven't seen something like aiconfig in generative AI developer tools yet.

    Our basic premise is that AI application development should be config-based, so you can track the prompts, models and model parameters being used more rigorously. Having this AI artifact then lets you iterate on it separately from your application code, and also set up evals that provide "test coverage" for the gen AI parts of your application.

    We were also inspired by the ipynb format for Jupyter notebooks, and you'll see parallels to that in the aiconfig format.

    Please ask any questions, and share your thoughts on config vs. code.

  • by jdwyah on 11/17/23, 8:08 PM

    I published https://www.prefab.cloud/blog/dynamic-config-for-openai-pyth... a few weeks ago as a way to do config of AI params like temperature, etc. But I've had a few people start using it for prompts as well and I really curious to see where it goes.

    In particular for 1. teams that have complex slow deploys, but want to change prompt now 2. when there are data analyst types doing the prompts and people don't want them to be able to "break things". 3. being able to alpha test / rollout / target new prompts easily.

    Definitely an interesting question whether prompts is code or configuration.

  • by activescott on 11/17/23, 7:16 PM

    Well done. Now that I've seen your getting started video (https://aiconfig.lastmileai.dev/docs/getting-started) this seems obvious. I feel really silly that I copy & paste from the browser back into a file just so I can save different prompts and results to see what I liked best. Thanks for sharing and for open sourcing!

  • by kordlessagain on 11/17/23, 9:51 PM

    I love this! Been working on something similar that uses containers and Jinja2 templates: https://github.com/kordless/SlothAI. A demo is running at https://mitta.ai. No docs yet and the auth system is getting an upgrade.

  • by smy20011 on 11/17/23, 6:18 PM

    Why not store them in code? Easier to read and source controlled.

  • by joshka on 11/18/23, 3:37 AM

    Relatedly similar project:

    https://promptfile.org/

  • by thatxliner on 11/17/23, 9:37 PM

    How does this compare to Ollama