by novaRom on 8/6/24, 12:41 PM with 4 comments
Why don't we see the emergence of local web search engines?
It seems feasible to handle large indices locally, even with periodic updates. Are there specific technical or commercial hurdles preventing this?
by _0ffh on 8/6/24, 12:56 PM
Also, local search is not scalable. Imagine if everyone, or even just a small fraction of end users, was trying to crawl the whole web for indexing.
But a couple of ideas that might actually work are:
1. A decentralised open source search engine, where the results are collated from a local index plus the indices of your connected peers, maybe plus a number of hops out to indirect connections. Would have to find a way to block users with tainted indices though, because that would quickly become part of the SEO market.
2. A local investigation engine without a central index to which you give a search query and which then trawls the net in a goal directed fashion to build up an index specifically with web sites that are interesting with regard to the search inquiry. You might have to leave it to run for a day or a couple, but you might be able to get good results with that, if done well.
by talldayo on 8/6/24, 12:58 PM