from Hacker News

MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

by qdequelen on 3/25/20, 4:01 PM with 113 comments

• by pqdbr on 3/26/20, 12:36 AM

  I'm impressed.

  I have a database with 15k documents, each with around 70 pages of text, HTML formatted.

  I'm using ElasticSearch currently, with the Searchkick gem.

  30 min playing with MeiliSearch. So far:

  - Blazing fast to index, like 10x more performant than using ElasticSearch / Searchkick;

  - Blazing fast to search, at least 3x faster in all my random tests so far;

  - Literally zero config;

  - Uses 140MB of RAM currently, while in my experience ElasticSearch would crash with anything less than 1GB, and needs at least 1.5GB to be usable in production.

• by heipei on 3/25/20, 8:25 PM

  I know the project doesn't claim it, but the title somewhat implies this: I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales. If anything, ElasticSearch for me has been one of the easiest pieces of infrastructure to operate, for me pretty much "zero-config". Let me elaborate: You can run ElasticSearch via Docker command-line, if you want a cluster you just supply IPs of the other nodes. Then you start indexing documents with simple HTTP calls. You can add or remove nodes at any time and don't have to do anything but to start another ElasticSearch instance. If you run out of space or performance just start another node. Everything needed for management, indexing, search is available through HTTP APIs, no tools needed.

  Clustered ElasticSearch has been rock-solid for me and I've used it in anger many times. The level of maintenance needed is close to zero, both initially and long-term. Compare that with the abysmal experience of setting up a sharded MongoDB cluster for example...

  Please enlighten me how ElasticSearch is "a lot of work to operate" (heard that one multiple times), and what you're comparing it to.

• by ghh on 3/25/20, 7:57 PM

  I wanted to mention Sonic [1] as another lightweight document indexing alternative written in rust, when I found MeiliSearch to provide a thoughtful comparison page [2]

  [1] https://github.com/valeriansaliou/sonic

  [2] https://docs.meilisearch.com/resources/comparison_to_alterna...

• by beagle3 on 3/26/20, 1:30 AM

  Mostly "made in Rust", but from the github readme[0] "MeiliSearch uses LMDB as the internal key-value store. The key-value store allows us to handle updates and queries with small memory and CPU overheads."; so a lot of the credit goes to LMDB, and safety implied by "made in Rust" is not, in fact, guaranteed.

  Not that I'm complaining - I love LMDB, and it's been rock solid and bug free in my experience (thanks, Howard!) - but it's low level C, not rust, and if you expect the certainty that Rust provides w.r.t to security, race conditions and leaks, be aware that you are not completely getting it.

  But other than that: Thanks! This looks like a great project!

  [0] https://github.com/meilisearch/MeiliSearch#how-it-works

• by ghayes on 3/25/20, 6:05 PM

  The goal of ElasticSearch, I always thought, was that it scales horizontally and can handle the loss of multiple nodes without availability- or data-loss. It's interesting to build a single-server replacement, and this can likely work for many use-cases, but it's definitely a different approach from ElasticSearch itself.

• by MuffinFlavored on 3/25/20, 6:15 PM

  The real power of Elasticsearch for me is the ability to filter logs by:

  1. exact match this nested JSON field (with support for lists of values)

  2. negative match this nested JSON field (with support for lists of values)

  coupled with the ability to filter by "timeframe", then pump it through to visualizations (tables/graphs) in Kibana

  MeiliSearch would be cool if it spoke the API Kibana expects from Elasticsearch

• by bryanrasmussen on 3/25/20, 8:27 PM

  ok I just looked through things a bit but the phrase 0 config worries me - first off I could conceivably run ElasticSearch with 0 configuration but then it needs to make decisions as to what types things are, and how things should be analyzed, and sometimes those decisions are not what I want.

  Often ElasticSearch makes a mistake in typing because the programmer has made a mistake in data format, if you fixed that mistake your data would now not fit the format that ElasticSearch has chosen for it (actually don't know if this is still a problem because it has been years since I have ran without all my fields being mapped first) but actually don't see how it couldn't be a problem.

  so theoretically if you didn't want to go through the trouble of defining a wrapping you could just reindex all your data fixed in such a way that ElasticSearch will choose a better type for individual fields but why would you do this?

  And I mean what does MelliSearch do? I wonder - because looking through this code here https://github.com/meilisearch/MeiliSearch/blob/master/meili... (and not being a rust guy my understanding of it is probably off) but it seems like maybe it is no configuration because it expects you to follow its semantics. Which to be fair lots of things do, at the base level, everything has a title, description, date.

  But if I have a domain with different or probably more advanced semantics what happens?

  Search Engines are generally configurable because you want to add other fields and rank hits in those fields higher than other things, or maybe do a specific search that only targets those fields - like say Brands based search.

  on preview: lots of other people with similar views it seems, I got maybe a bit ranty just because the title sets me off when it just is so wrong it even seems like lying.

• by manigandham on 3/26/20, 1:39 AM

  Awesome, glad to see all the competition in the search space now. There are other projects like Sonic, Tantivy, Toshi and more that have more functionality if you need alternatives.

  Here's a public list of search projects (in rust, c, go): https://gist.github.com/manigandham/58320ddb24fed654b57b4ba2...

• by seemslegit on 3/25/20, 6:28 PM

  Hardly an "alternative to Elastic search" if only because the later is scalable beyond a single machine.

  This overhyped description coupled with on-by-default analytics suggests to me MeiliSearch should be dismissed regardless of potential usefulness or technical merit.

• by time0ut on 3/25/20, 9:18 PM

  Nice. This looks promising. Very clean API. I like the focus on a narrow use case.

  Do you have any information on security topics like using TLS, client authentication, etc?

• by otterley on 3/25/20, 8:38 PM

  MeiliSearch appears to be more of an alternative to Lucene than it is to Elasticsearch. Lucene is the search engine that runs on a single instance; ES is the horizontally-scalable distribution and aggregation layer atop the instances. Absent a similar aggregation layer, MeiliSearch isn't "elastic" as the comparison implies.

• by nreece on 3/26/20, 1:41 AM

  Looks pretty good. The single filter approach is restrictive though.

  We're currently leaning towards Manticore Search[1], which is a fork of Sphinx Search[2].

  [1] https://manticoresearch.com [2] http://sphinxsearch.com

• by Bedon292 on 3/25/20, 7:03 PM

  While this might be an alternative for that one specific use case (search bar), it does not feel like a viable alternative to ES. I am sure it is great at that specific case, and don't want to knock them on that. But, I have never used ES for a simple search like they are. when I use ES, I want to store billions of records redundantly and search them by text, time, and/or location. And then create visualizations with the results.

  When I first read the title I thought it might be a Rust based Lucene engine or something, and thought that would be pretty cool. Though no idea how that would work. On its own, this is a pretty nifty little tool, however I think the framing as an ES alternative is what feels wrong to me, and apparently others in the comments as well.

• by dalore on 3/26/20, 10:43 AM

  Wow to see this popup is strange as I was just implementing this yesterday.

  It is blindingly fast and easy to setup.

• by mleonhard on 3/26/20, 6:26 PM

  > MeiliSearch can serve multiple indexes, with different kinds of documents, therefore, it is required to create the index before sending documents to it.

  https://github.com/meilisearch/MeiliSearch#create-an-index-a...

  Indexes are config. This is not really zero-config if you require API calls before it can receive data.

  Also, there's nothing about TLS or access control. These will be required for any production deployment. At the minimum, let us specify a TLS key.pem and cert.pem file and create write-only and read-only access tokens.

• by karterk on 3/26/20, 1:45 AM

  If you are looking for alternatives, check out Typesense as well:

  https://github.com/typesense/typesense

• by maxpert on 3/26/20, 4:04 AM

  How does it compare to Sonic https://github.com/valeriansaliou/sonic

• by throw03172019 on 3/25/20, 6:27 PM

  Are the documents stored on disk or only in memory?

• by niyazpk on 3/26/20, 5:17 PM

  Does anyone know if this supports bulk indexing? My team has a lot of data in S3 in parquet format. (We could change the format to something else if that helps).

  It would be really nice to be able to point tools like MeilliSearch or ElasticSearch to a data location and have it index all the data without me writing code to send individual records to the API.

• by dzonga on 3/26/20, 5:32 PM

  looks really easy to use. will use this instead of resorting to Postgres full text search for my next app(s)

• by bradrobertson on 3/27/20, 3:29 PM

  Looks promising! Are there any docs coming on a production ready setup? Reading below it looks like you're working on high availability, but even in the single machine scenario, do you have recommendations for persistence, fault tolerance etc?

• by throw03172019 on 3/25/20, 7:23 PM

  We use Algolia and use the public API keys with search filters encoded so they can only search their data (I.e. account_id:123)

  Is there anything similar here? Otherwise all the queries need to go through our servers first to ensure the filter is present.

• by udfalkso on 3/25/20, 8:26 PM

  Sounds more like a potential alternative to Sphinx than Elastic Search.

  sphinxsearch.com/

• by eliseumds on 3/25/20, 7:56 PM

  Pretty heavy user of ES here, and one cannot compare the two products.

• by kvz on 3/25/20, 9:36 PM

  Is there already a browser library that can talk to MeiliSearch?

• by dhruvkar on 3/26/20, 5:37 PM

  I've never used elasticsearch and only had a brief toy project with Algolia. The demo on the github repo looks awesome.

  Can this run on top of my postgres database?

• by bberenberg on 3/26/20, 7:04 PM

  Does any tool in this category (This, Elastic, or whatever else) support something like permissions on a per document level?

• by social_quotient on 3/25/20, 11:37 PM

  Thanks for including performance metrics right up front!