from Hacker News

What should the next search engine look like?

by bigcloud1299 on 10/1/19, 12:32 AM with 14 comments

I am working on researching what should a next generational product and content search engine should look like that uses AI and ML.

Being new to this space, I would greatly appreciate your sharing your thoughts and any documents that i can use as reference.

  • by tlack on 10/1/19, 1:46 AM

    Disclaimer: Startup in this space, highly biased, shameless pluggery.

    I think about this subject a lot. Three key innovations we'll see in the "next wave" of search engines that flood in to replace stinky incumbents (won't name any names here)

    1. New user interfaces. My effort, Glorp, attempts to transition from the "link dump" style of results page to something akin to Instagram. Faster to scan, fewer surprises, more friendly for kids and old folks. AR/VR will have some unusual needs. Audio - Alexa and Siri. We need search (in different forms) everywhere. [1]

    2. New business models. Privacy is in, advertising is out. That changes everything about the model. We'll see engines with tons of niche features, like link collections, recorded search journeys, curated subject matter experts, family accounts with parental controls, and, most importantly, monthly fees.

    3. New search mechanisms: ML embeddings. Search by image similarity, video similarity, subject matter similarity. Machine learning models transform different types of content into convenient embedding vectors. BERT gives you a few hundred floats to represent the meaning of a sentence. ImageNet, the same, for images. Once you begin to build search tooling around the notion of document embeddings, rather than just "input text," we'll be able to help users discover things in a much broader set of scenarios. [2]

    Hit me up, email in profile

    [1] https://glorp.co/Search/search%20engine%20user%20interfaces

    [2] https://glorp.co/Search/search%20engine%20vector%20embedding...

  • by pjbk on 10/1/19, 4:15 AM

    Does anyone remember the old Kartoo graphical/relational search engine? (Not its current hijacked reincarnation at kartoo.com)

    It was not very practical as a generic search engine, but I found some very interesting links following the maps of related terms. I thought that kind of intuitive user interface would be improved and become popular, but they never did.

  • by milsebg on 10/1/19, 4:13 PM

    I don't know if there will be a 'next search engine'. From the days before Google, search engine in general haven't changed much.

    In the beginning there was just string matching, word stemming followed, then we had synonyms and are getting more towards NLP for getting the user intend. Also, if the communication with the engine is via a text box or via a voice command or a live smartphone picture (for augmented reality) does not really matter.

    So, what can change is just the "smartness" of information retrieval at the backend and the ranking factors (from freshness to PageRank to whatever).

    Given the fact that the market power of Google actually shapes the internet by introducing their best practices (like AMP or schema.org) on webmasters on a large scale, I doubt that there will be one company with the big breakthrough in information retrieval like Google was. Maybe this field is now more or less saturated for the majority of use cases that it will only evolve slowly and not revolutionary.

  • by sova on 10/1/19, 2:38 PM

    UI ideas: + Infinite trail of breadcrumbs (pages visited) that you could potentially share with others (show how you found the info based on path)

    + Tree-based navigation (go back a page and overwrite your cached future? no thanks)

    + some sort of warmer/colder haptic feedback could be cool if there's a convergence on results for specific terms

    + some useful topic overview of things I searched in the last 1 year, 2 years, 3 years, etc. See where my mind has been.

  • by d-sc on 10/1/19, 4:21 AM

    Idk, I’ve always looked for a search engine that prioritizes good content over pages with ads. Like, there’s 7 billion people on this planet, are you telling me there is no one that has a cookie recipe that nicely fits on one page without going through oodles of advertising.
  • by freediver on 10/1/19, 12:43 AM

    Think about the business model first. This is a good place to start.

    https://twitter.com/ianbremmer/status/966677755424313344?s=2...

  • by tucaz on 10/2/19, 2:45 AM

    I’d like one that works and finds the stuff I’m looking for. How about that?

    Not sure I (or pretty much anyone else) care how it does that.

  • by streetcat1 on 10/1/19, 3:04 AM

    I will give you a clue, or rather a framework for thinking. Then you can elaborate more.

    So the imbalance today exists because the search engine uses AI against humans.

    The next phase will be (or should be) AI against AI.

    The human UI of the search engine will not be used by humans but by an AI representing the human.

    If you follow this paradigm, you can see how powerless the centerl search engine becomes.

  • by mrfusion on 10/1/19, 9:09 PM

    Actually google but not gamed by seo would be next generation.