from Hacker News

Quickwit 0.8: Indexing and Search at Petabyte Scale

by vvoyer on 3/19/24, 2:56 PM with 28 comments

  • by dracyr on 3/23/24, 6:18 AM

    Never had the chance to use Quickwit at a $DAYJOB (yet?), but I really appreciate the fact that it scales down quite well too. Currently running it on my homelab, after a number of small annoyances using Loki in a single-node cluster, and it's been working very well with very reasonable resource usage.

    I also decide to use Tantivy (the rust library powering/written by Quickwit) for my own bookmarking search tool by embedding it in Elixir, and the API and docs have been quite pleasant to work with. Hats of to the team, looking forward to what's coming next!

  • by godber on 3/23/24, 2:45 PM

    We did some experimentation with quickwit about a year ago, writing about 1m docs/second of data into it for several months. It worked well and was pretty straight forward to learn and operate. If we didn’t also manage our own S3/Ceph it might be a big win, once feature complete. It’s definitely worth a look.

  • by halvorbo on 3/23/24, 11:20 AM

    Amazing to see how far Tantiviy has come. Remember using and making some smaller contributions to this 3 years ago - slop to phrase queries for example. Curious how the design has changed to enable large scale production usage.

  • by up2isomorphism on 3/23/24, 6:38 AM

    13.4GB/s with 200x6 vcpus, gives 11MB/s per core, it is good but hard to say impressive.

  • by arisudesu on 3/23/24, 5:36 PM

    musl support would be highly appreciated.