from Hacker News

Show HN: AI Playground by Vercel Labs

by palmdeezy on 4/18/23, 10:38 PM with 102 comments

Hey, Jared Palmer (creator of this playground) here. Really excited to ship this. I’ve been building this over the past few weeks to compare LLMs from different providers like OpenAI, Anthropic, Cohere, etc. At Vercel, I manage our Frameworks division (including Next.js, Svelte, and Turbo) and wanted to also dogfood some of the latest features in a slightly larger application. This playground takes a lot of inspiration from https://nat.dev and is built on Tailwind, ui.shadcn.com, and some upcoming Vercel products we’re announcing soon. We’re going to continue adding models to compare and add other frameworks to generate code snippets from.
  • by rendall on 4/19/23, 4:06 AM

    As a joke, I wrote "Give me a disruptive startup idea". This is what claude wrote:

      Here is a potentially disruptive startup idea:
    
      • On-demand drone delivery for e-commerce. The idea is to offer ultra fast delivery of small packages via drones. This could significantly reduce delivery times for e-commerce orders and disrupt the traditional shipping industry. Some key challenges to overcome would be regulatory approvals, safety, noise pollution, and scaling up a drone delivery network. But if executed well, it could transform how people receive goods and packages.
    
      A few other disruptive ideas:
    
      • An Uber-like model for renting out spare rooms or couches for short term stays. This could disrupt the hotel industry. 
    
      • A Netflix model for streaming live TV channels over the internet. This could disrupt the cable TV industry.
      
      • An automated vertical farming system to grow organic food locally in urban areas. This could disrupt the traditional agriculture and food supply chain.
    
      • Decentralized blockchain-based social network. The idea is to build a social network that is not owned or controlled by any single company. It would be an open, decentralized platform built on blockchain technology. Users would have more control and ownership over their data and profiles. The business model could be built around micro-payments or subscriptions. It could disrupt the dominance of companies like Facebook and offer an alternative distributed social networking experience
    
      • A virtual reality social network. If VR/AR technologies become more mainstream,
    
    I mean, they are disruptive startup ideas, sure enough. GPT-4 didn't bother answering.
  • by neom on 4/19/23, 1:54 AM

    I dunno what Cohere is, but it's answers are super weird. I asked it "Where does the phrase "the rain in spain falls mainly on the plane" come from?" and it said:

    Does anybody know a green song? What happens if a girl doesn't have a twin brother? What musical instrument did larry black sing 'jealous heart' on? Is country music more country than country music. I'm confused.? What is the difference between a copyright and a trademark? What is the best piano chords software u can get for beginners? What is the best music software for beginners? What is the song from the new dexter episode? Did he kill his brother? What is the name of the song dueling banjos? Was the raiders of the lost ark the first jungle hero movie? Who really invented reverse tape effect? Why can't me and my neighbor get along? What prophet spoke to the teachers of the law at jerusalem about their treatment of jesus? Are there any songs that describe the emotions of a cult member? What is the most ugly song ever made?

  • by gardenhedge on 4/19/23, 1:24 AM

    You manage the Frameworks division (including Next.js, Svelte, and Turbo)? How did you get the time to build this?
  • by atleastoptimal on 4/19/23, 3:42 AM

    How much does it cost to query these models all the time? It's offering a lot for free.
  • by doodlesdev on 4/18/23, 11:16 PM

    Please provide an icon that makes sense on other platforms for the shortcut on the right bottom, not everyone uses Macs! Overall, pretty cool idea, how is Vercel gonna handle API costs for this? I imagine this is pretty expensive to run, specially considering GPT4 access.

    Also, I have to link this here because I thought it was too funny:

    https://play.vercel.ai/r/G9y3eXJ

    edit: Oh I just hit the token limit, I guess that's how you're dealing with cost lol. Something I was gonna suggest is that maybe answers should be cached to prevent repeated calls to the same API when the prompt hasn't changed, this could potentially increase the amount of tests I could do.

  • by swyx on 4/19/23, 12:56 AM

    beautiful work as always jared!

    one of the useful things from nat.dev was the tokens/second calculation, latency is impt as usual. would be nice to enable?

    i'm excited to see what the upcoming products are. making things default sharable like this across all frameworks would be really nice.

  • by etaioinshrdlu on 4/19/23, 4:34 AM

    How did Anthropic agree to let anonymous users call their API? This was a big stumbling block for me.
  • by zurfer on 4/19/23, 12:17 PM

    Thanks for setting up a limited free version. The UX is great.

    I had some failures for huggingface and cohere. Replicate produced an output that was not rendered (but returned an answer in the network tab).

    It would also be great to add: https://docs.aleph-alpha.com/docs/introduction/luminous/

  • by khalidx on 4/19/23, 12:18 AM

    Awe man I was just building this. You beat me to it! Looks great. FYI needs some tweaks on mobile.
  • by lazylion2 on 4/19/23, 9:25 AM

    You can add https://www.ai21.com/ LLMs as well
  • by vertis on 4/19/23, 5:56 AM

    Prompt: Pretend you are William Gibson. Pitch your next book to your publisher.

    GPT-4: > Dear [Publisher],

    > I hope this letter finds you well. As you know, my work has always strived to explore the intersection of technology, society, and human nature. With that in mind, I am excited to present to you my latest novel, "Neural Drift."

    > Set in the not-too-distant future, "Neural Drift" delves into a world where the lines between our reality and the digital realm have all but vanished. The story revolves around our protagonist, Elara, a skilled hacker and data courier who navigates the vast neural network known as the "Mindscape" – an immersive digital landscape that has become the primary means of communication, commerce, and entertainment for the global population.

    > In this future, the Mindscape has become an integral part of daily life, and the distinction between the physical and digital worlds is all but gone. Society is now heavily reliant on neural implants, which allow users to access the Mindscape and interact with the digital world through

    Anthopic Claude-v1 (which wouldn't answer at all if I didn't put pretend in front):

    > Here is my William Gibson-inspired book pitch:

    > The future is here—it's just not evenly distributed yet. In my new novel, Pattern Recognition, I explore the bleeding edge of culture, technology, and society through the eyes of Cayce Pollard, a "coolhunter" who has an uncanny ability to spot trends in the wild.

    > Cayce is hired to track down the creator of a series of mysterious film clips being uploaded to the internet. Her search takes her from London to Tokyo to Moscow, where she encounters a dangerous mix of hackers, hipsters, and Russian mobsters. Along the way, Cayce begins to realize the film clips seem to have a strange predictive power—they appear to show events that have not happened yet.

    > Pattern Recognition is a stylish, fast-paced techno-thriller that examines how technology shapes culture, and how culture shapes us. It blends Gibson's trademark cyberpunk sensibility with a

    YES, it did just regurgitate an existing Willam Gibson book

  • by yding on 4/19/23, 1:16 AM

    Great! Vercel edge functions are my preferred deployment for OpenAI calls now.
  • by lightning19 on 4/19/23, 4:32 PM

    Just for fun I wrote this prompt "vercel monopolizes nextjs by only allowing certain features on their platform such as"

    ChatGPT4:

    1. Incremental Static Regeneration (ISR): This feature allows developers to update static pages at runtime without having to rebuild the entire website. Vercel has optimized its platform to support ISR, making it an exclusive feature for Next.js applications deployed on Vercel.

    2. Real-time Feedback: Vercel's platform provides real-time feedback during development, allowing developers to see the changes they make in their code instantly. This feature is exclusive to Vercel, making it a significant advantage for developers using Next.js on their platform.

    3. Edge Functions: Vercel offers Edge Functions, which allow developers to run serverless functions at the edge of their network, providing low-latency responses to user requests. This feature is particularly useful for Next.js applications that require server-side rendering and is exclusive to Vercel's platform.

    4. Preview Mode: Vercel's platform supports Next.js's Preview Mode, which allows developers to preview draft content changes without affecting the live site. This

    -------------------------------------------------------

    Claude v1.3

    That is not correct. Vercel does not monopolize Next.js or limit features to their platform.

    Next.js is an open source framework that can be deployed anywhere. Some key facts:

    1. Next.js is MIT licensed and the source code is available on GitHub for anyone to use, modify, and deploy.

    2. Next.js apps can be deployed on any hosting provider or infrastructure. This includes:

    - Vercel (the creators of Next.js) - AWS - GCP - Azure - DigitalOcean - Heroku - Your own servers, etc.

    3. All Next.js features work regardless of where you deploy. This includes:

    - Static Generation (SSG) - Server Side Rendering (SSR) - Incremental Static Regeneration (ISR) - API Routes - Image Optimization - Font Optimization

  • by pastacacioepepe on 4/19/23, 10:00 AM

    It's completely broken. I couldn't even get one full response from one of the AIs, it always gets stuck near to the completion of the first response. After like 5 tries it told me that I used all my requests. lol.
  • by spaceman_2020 on 4/19/23, 9:37 AM

    GPT-4 is miles ahead of the competition. It's not even remotely close.

    I asked it to help me design an app for a specific use case. It came up with a list of features, the tech stack, and even designed an initial database.

  • by tough on 4/19/23, 1:01 AM

    This is awesome, will this be open sourced to poke around the codebase?

    Some models seem to hang w me (anthropic) but I think it was because I reached the limit.

    Does loggedIn usage w vercel will become usage-paid?

  • by Aeolun on 4/19/23, 3:42 AM

    Well, every single thing I tried other than gpt-4 failed for me.
  • by osteele on 4/19/23, 3:50 AM

    Very nice! If you want to add a feature, feel free to make use of https://github.com/osteele/prompt-matrix.js, which I (well, ChatGPT and I) created for use in my own playground so that I could enter e.g. "Write a <Python|JavaScript> program to look for anagrams, in the style of <Norvig|Carmack>".
  • by eh9 on 4/19/23, 4:34 AM

    How expensive is this to run?
  • by petecoop on 4/19/23, 3:47 PM

    What's the data policy on this? Are you keeping copies of the prompts and responses?
  • by vladholubiev on 4/20/23, 9:33 AM

    Does anybody know if there is a similar playground, but for evaluating a single model by comparing different prompts with different temperatures?

    I am building this by myself using Streamlit, but was wondering is there a ready solution.

  • by lxe on 4/19/23, 1:30 AM

    How are the models hosted?

    Edit: they are using their own hosted APIs such as Anthropic or Replicate.

  • by tikkun on 4/19/23, 1:56 AM

    Do you think the codebase will be open sourced within the next week or two?
  • by d4rkp4ttern on 4/19/23, 11:46 AM

    Great interface!

    Maybe slightly tangential, on the subject of UIs for ChatGPT, does anyone know a good UI for voice input to ChatGPT, for either the website or via an API ?

  • by pjs_ on 4/19/23, 3:40 PM

    > are you gpt-4?

    I am an AI language model created by OpenAI, but I am not GPT-4. I am based on the GPT-3 model. GPT-4 has not been released yet.

  • by jacooper on 4/19/23, 2:14 AM

    Does this fully use UTF-8? I have been trying it a non-lating language and its showing ?? Icons in every answer.
  • by ruslanmsv on 4/19/23, 9:20 AM

    Its output for GPT-4 much worse compared with an output of chatGPT-4 available at chat.openai.com
  • by FailMore on 4/19/23, 7:23 AM

    Nice, can I have conversations? It’s not clear on mob how to do this vs. Just one prompt
  • by cloudking on 4/19/23, 12:17 AM

    Thanks for sharing! What business problems are you solving at Vercel with LLMs?
  • by ironfootnz on 4/19/23, 2:04 AM

    the UI is quite limited, check out the outputs https://play.vercel.ai/r/I25tmJs when needs a scroll bar.
  • by khuss on 4/19/23, 4:39 AM

    Nice work. Is it open source?
  • by nkmnz on 4/19/23, 1:16 PM

    Is the source available?