from Hacker News

Llama-agents: an async-first framework for building production ready agents

by pierre on 6/28/24, 4:54 PM with 28 comments

  • by ldjkfkdsjnv on 6/28/24, 6:51 PM

    These types of frameworks will become abundant. I personally feel that the integration of the user into the flow will be so critical, that a pure decoupled backend will struggle to encompass the full problem. I view the future of LLM application development to be more similar to:

    https://sdk.vercel.ai/

    Which is essentially a next.js app where SSR is used to communicate with the LLMs/agents. Personally I used to hate next.js, but its application architecture is uniquely suited to UX with LLMs.

    Clearly the asynchronous tasks taken by agents shouldnt run on next.js server side, but the integration between the user and agent will need to be so tight, that it's hard to imagine the value in some purely asynchronous system. A huge portion of the system/state will need to be synchronously available to the user.

    LLMs are not good enough to run purely on their own, and probably wont be for atleast another year.

    If I was to guess, Agent systems like this will run on serverless AWS/cloud architectures.

  • by cheesyFish on 6/28/24, 4:59 PM

    Hey guys, Logan here! I've been busy building this for the past three weeks with the llama-index team. While it's still early days, I really think the agents-as-a-service vision is something worth building for.

    We have a solid set of things to improve, and now is the best time to contribute and shape the project.

    Feel free to ask me anything!

  • by dr_kretyn on 6/29/24, 12:18 AM

    Can't really take it seriously seeing "production ready" next to a vague project that has been started three weeks ago.

  • by gmerc on 6/29/24, 10:03 AM

    How do you overcome compounding error given that the average LLM call reliability peaks well below 90%, let alone triple/9

  • by jondwillis on 6/28/24, 10:54 PM

    why use already overloaded “llama”

  • by k__ on 6/28/24, 6:36 PM

    I have yet to see a production ready agent.

  • by williamdclt on 6/28/24, 7:37 PM

    I must be missing something: isn’t this just describing a queue? The fact that the workload is a LLM seems irrelevant, it’s just async processing of jobs?