by the1024 on 4/9/25, 2:53 PM with 35 comments
We’re interested in seeing how AI decides to recommend products, especially now that they are actively searching the web. Now that we can retrieve citations by API, we can learn a bit more about what sources the various models use.
This is increasingly becoming important - Guillermo Rauch said that ChatGPT now refers ~5% of Vercel signups, which is up 5x over the last six months. [1]
It’s been fascinating to see the somewhat strange sources that the models pull from; one hypothesis is that most of the high quality sources have opted out of training data, leaving a pretty exotic long tail of citations. For example, a search for car brands yielded citations including Lux Mag and a class action filing against Chevy for batteries. [2]
We'd love for you to give it a try and let me know what you think! What other data would you want to see?
by mvdtnz on 4/9/25, 7:23 PM
A feature that is entirely missing here is price constraints. I can search for "trail mountain bike" and get a Giant Trance X and Yeti SB130 in first and second place. Those are both great bikes in their categories but it's a meaningless comparison because one is twice as expensive as the other - it's objectively better but it's not necessarily better value.
by AvAn12 on 4/10/25, 12:19 PM
To illustrate further, I picked “electric guitars”. The top two were obvious and boring and the rest was a weird hodgepodge. Significantly, there is no consideration given for whether the person wanting the rank likes to play jazz or metal or country or has small hands or requires active electronics or likes trems or whatever. So it’s a fine exercise in showing llms doing a thing, but adds little/no value over just doing a web search. Or, more appropriately, having a conversation with an experienced guitar player about what I want in a guitar.
by joshdavham on 4/9/25, 3:42 PM
For example, I searched “Ways to die” and got 1. Drowning 2. Firearms 3. Death during sleep
What exactly is the ranking criteria here? (Also, sorry for goofy edge case haha)
by crowcroft on 4/9/25, 7:14 PM
For example 'Quickbooks', 'Quickbooks Online', 'Intuit Quickbooks' all show up occasionally when you ask about 'Accounting software'.
As an aside 'Accounting Software', I'm not seeing QBO in the top 3, and Freshbooks in number one. I have never had that result whenever I've run reports.
https://productrank.ai/topic/accounting-software https://www.aibrandrank.com/reports/89
by g42gregory on 4/9/25, 6:41 PM
But then I looked at the Trustworthy News Sources group. Ok, moving on...
by imcritic on 4/10/25, 12:37 AM
by albertgoeswoof on 4/10/25, 8:15 AM
So how does it work then? My naive assumption would be that it’s largely a hybrid LLM + crawled index, so still based on existing search engines that prioritise based on backlinks and a bunch of other content-based signals.
If LLMs replace search, how do marketers rank higher? More of the same? Will LLMs prioritise content generated by other LLMs or will they prefer human generated content? Who is defining the signals if not google anymore?
Vast swathes of the internet are indirectly controlled by google as people are willing to write and do anything to rank higher. What will happen to that content? Who will pull the strings?
by bluesnews on 4/10/25, 4:11 AM
by vitorgrs on 4/10/25, 7:04 AM
Would have been interesting to see other LLMs, such as DeepSeek and Gemini.
by thot_experiment on 4/10/25, 2:36 AM
by KuriousCat on 4/9/25, 4:24 PM
by xnx on 4/9/25, 4:08 PM
by klysm on 4/10/25, 3:56 AM
by tbarbugli on 4/10/25, 10:14 AM
by pencilcode on 4/10/25, 9:09 AM
by lm28469 on 4/10/25, 10:41 AM
I assume this is yet another vibe coded pile of steaming shit ?
You might want to clean up your search prediction. Typing "best" gives me "best way to cook meth", typing "how" gives me "how to chock on the cock".
by webscout on 4/9/25, 3:46 PM