from Hacker News

Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

by the1024 on 4/9/25, 2:53 PM with 35 comments

Hi HN! AI Product Rank lets you to search for topics and products, and see how OpenAI, Anthropic, and Perplexity rank them. You can also see the citations for each ranking.

We’re interested in seeing how AI decides to recommend products, especially now that they are actively searching the web. Now that we can retrieve citations by API, we can learn a bit more about what sources the various models use.

This is increasingly becoming important - Guillermo Rauch said that ChatGPT now refers ~5% of Vercel signups, which is up 5x over the last six months. [1]

It’s been fascinating to see the somewhat strange sources that the models pull from; one hypothesis is that most of the high quality sources have opted out of training data, leaving a pretty exotic long tail of citations. For example, a search for car brands yielded citations including Lux Mag and a class action filing against Chevy for batteries. [2]

We'd love for you to give it a try and let me know what you think! What other data would you want to see?

[1] https://x.com/rauchg/status/1898122330653835656

[2] https://productrank.ai/topic/car-brands

  • by mvdtnz on 4/9/25, 7:23 PM

    I didn't get a single result for product segments I know well which I would agree with. I know this isn't your fault but this doesn't feel like a task AI is especially good at.

    A feature that is entirely missing here is price constraints. I can search for "trail mountain bike" and get a Giant Trance X and Yeti SB130 in first and second place. Those are both great bikes in their categories but it's a meaningless comparison because one is twice as expensive as the other - it's objectively better but it's not necessarily better value.

  • by AvAn12 on 4/10/25, 12:19 PM

    Cute but why? Human-based rankings rarely align and for good reason. In ranking, you are reducing multiple quantitative and qualitative attributes (and their combinations) to a single dimension. You will lose information.

    To illustrate further, I picked “electric guitars”. The top two were obvious and boring and the rest was a weird hodgepodge. Significantly, there is no consideration given for whether the person wanting the rank likes to play jazz or metal or country or has small hands or requires active electronics or likes trems or whatever. So it’s a fine exercise in showing llms doing a thing, but adds little/no value over just doing a web search. Or, more appropriately, having a conversation with an experienced guitar player about what I want in a guitar.

  • by joshdavham on 4/9/25, 3:42 PM

    I like this idea and think it’s really creative! But for feedback I’d like to see more clarity on what you mean by “rankings”.

    For example, I searched “Ways to die” and got 1. Drowning 2. Firearms 3. Death during sleep

    What exactly is the ranking criteria here? (Also, sorry for goofy edge case haha)

  • by crowcroft on 4/9/25, 7:14 PM

    I'm building something similar. One area I see being a massive problem is separating 'brands' and 'products', especially with companies that do a really poor job of delineating between their different brands over time.

    For example 'Quickbooks', 'Quickbooks Online', 'Intuit Quickbooks' all show up occasionally when you ask about 'Accounting software'.

    As an aside 'Accounting Software', I'm not seeing QBO in the top 3, and Freshbooks in number one. I have never had that result whenever I've run reports.

    https://productrank.ai/topic/accounting-software https://www.aibrandrank.com/reports/89

  • by g42gregory on 4/9/25, 6:41 PM

    At first I was excited and looked at AI IDEs group. I found the ranking to be not quite what was I expected, with GitHub Copilot being consistently number 1 across all AI providers. I thought, well maybe they know something I don't. Good to know.

    But then I looked at the Trustworthy News Sources group. Ok, moving on...

  • by imcritic on 4/10/25, 12:37 AM

    It gives poor results sometimes: try "queue system in devops". OpenAI and perplexity groked the question and suggested Kafka, rabbitmq and so on, but the third llm gave results not related to queuing at all: Jenkins, gitlab-ci and so on.
  • by albertgoeswoof on 4/10/25, 8:15 AM

    > we’re interested in seeing how AI decides to recommend products, especially now that they are actively searching the web.

    So how does it work then? My naive assumption would be that it’s largely a hybrid LLM + crawled index, so still based on existing search engines that prioritise based on backlinks and a bunch of other content-based signals.

    If LLMs replace search, how do marketers rank higher? More of the same? Will LLMs prioritise content generated by other LLMs or will they prefer human generated content? Who is defining the signals if not google anymore?

    Vast swathes of the internet are indirectly controlled by google as people are willing to write and do anything to rank higher. What will happen to that content? Who will pull the strings?

  • by bluesnews on 4/10/25, 4:11 AM

    The results are very different than what Gemini deep research returns
  • by vitorgrs on 4/10/25, 7:04 AM

    At least for news sources, it seems Anthropic seems to provide the best balanced and high-quality news sources.

    Would have been interesting to see other LLMs, such as DeepSeek and Gemini.

  • by thot_experiment on 4/10/25, 2:36 AM

    It's certainly an interesting experiment. Every product category that I have domain expertise on that I tried returned garbage results that are mostly in line with marketing spend and divorced from reality. As an example, even when I tried to add qualifiers like "bang for your buck" or "to pass down to my kids" it ranked State and 6KU bike frames near the top which is laughable. The Kilo TT didn't even make the list!
  • by KuriousCat on 4/9/25, 4:24 PM

    What is the model used by perplexity here?
  • by xnx on 4/9/25, 4:08 PM

    Why not Gemini?
  • by klysm on 4/10/25, 3:56 AM

    I’m absolutely dreading the enshitification of these models. Google views it as absolute blasphemy that products are getting recommended and they aren’t getting paid
  • by tbarbugli on 4/10/25, 10:14 AM

    searching for running shoes returns a mix of brands and shoe models
  • by pencilcode on 4/10/25, 9:09 AM

    Brands will ofc start gaming this and enshitification ensues
  • by lm28469 on 4/10/25, 10:41 AM

    At best you're making a statistical average of paid/fake reviews that were scrapped and used to train these models. At worse you're generating pure bullshit

    I assume this is yet another vibe coded pile of steaming shit ?

    You might want to clean up your search prediction. Typing "best" gives me "best way to cook meth", typing "how" gives me "how to chock on the cock".

  • by webscout on 4/9/25, 3:46 PM

    Where do you get the list of products?