from Hacker News

In-memory database Redis wants to dabble in disk

by digitalnalogika on 10/19/23, 9:26 AM with 58 comments

• by mrinterweb on 10/19/23, 9:31 PM

  I was confused because I know Redis already offers persistence. https://redis.io/docs/management/persistence/ The main difference here is how its runtime will shift from runtime data only in RAM to not storing as much data in RAM and looking up on disk. Current Redis persistence looks like the runtime is stored in RAM and changes are periodically or append only persisted to disk. I hope I got that right.

• by hivacruz on 10/19/23, 9:03 PM

  For those interested in an alternative to Redis on disk, compatible with the Redis protocol, have a look on Kvrocks[1] which was recently accepted in the Apache foundation. It is based on RocksDB and works quite nicely for us.

  [1]: https://github.com/apache/kvrocks

• by donatj on 10/20/23, 5:32 AM

  > It is part of the drive to make Redis "more like your classic database," he said. In the future, support for natural language queries and enhanced vector and feature store capabilities will be added. This initiative aligns with Redis's ambition to be seen as more than just a fast, albeit expensive, cache

  This sounds to me like not understanding your place in the market or why people use your product. I can name half a dozen classic SQL databases off hand. I can’t name a tool that competes with Redis at their niche. Why aim for an already oversaturated market when you already have a good profitable niche.

• by whalesalad on 10/19/23, 9:40 PM

  > One main criticism of Redis had been its lack of support for SQL, the ubiquitous query language. Trollope said that was fixed now. A module RediSQL is available on GitHub.

  I think redisql has been supplanted by zeesql - https://zeesql.com/

• by ohnoesjmr on 10/19/23, 11:42 PM

  I've tried Redis-on-flash (enterprise version), and performance was very poor. Its just lsmt underneath and suffers from all the usual lsmt issues.

  In the end we went with Aerospike which actually while backed by NVMe, outperforms redis backed by memory. Aerospike in memory is completely absurd speeds and throughput. Sadly the setup is a bit weird and query model is a bit wonky but we worked around it. A bit sad on how expensive the enterprise version is.

• by karmakaze on 10/19/23, 5:22 PM

  They mention both flash and disk, which finally gets explained as tiers 2 and 3. Also less clear though I assume this applies to their hosted service offering and not the core software.

  > "We can take the lesser-used data that hasn't been touched in a while and shuttle it off to flash where it can sit for a while. When the user comes back eventually, it's very easy for us to seamlessly move it from flash back into memory. And that allows the company to save costs," he said.

  > Redis is now planning to extend the concept to disk-based memory to offer support for a three-tiered architecture.

• by iot_devs on 10/20/23, 8:16 AM

  How bad is Redis with a swap file on SSD?

  I would expect it to be a very reasonable middle ground that requires almost no work and it is already enough and ready for most use cases.

• by otterley on 10/20/23, 2:41 AM

  Anyone know how their plans differ from the Virtual Memory plans that antirez had about 13 years ago, implemented in Redis 2.6, and then (wisely) abandoned?

  https://redis.io/docs/reference/internals/internals-vm/

• by tehbeard on 10/20/23, 5:51 AM

  How would the example of a scoreboard work with tiered memory?

  Surely you need the sorted set in memory to derive rank/manipulate it etc?

  Or would you have to shard it and add other elements on top in app code to focus on only a subsection of the data?

• by remram on 10/20/23, 2:52 PM

  I think I would rather they added a write-ahead-log to make writes durable, than disk lookup to support bigger-than-RAM datasets (slowly).

• by andrelaszlo on 10/20/23, 8:30 AM

  It's not mentioned in the article but this is Redis Enterprise which is more like a superset of open source Redis.

• by infomaniac on 10/20/23, 4:34 AM

  memcached has recently gained the ability to spill to disk: https://github.com/memcached/memcached/wiki/Extstore

  we recently implemented this to grow our caches to >50TB

• by fumeux_fume on 10/20/23, 6:20 AM

  Redisk... Sounds like a good April fools joke

• by reconditerose on 10/19/23, 3:13 PM

  It is part of the drive to make Redis "more like your classic database," he said. In the future, support for natural language queries and enhanced vector and feature store capabilities will be added.

  Seems way off the mark for why people use Redis. Developers use it precisely because it's not like their classic databases.

• by 9dev on 10/19/23, 9:14 PM

  What I actually need is a better way to bootstrap a cluster using Docker containers …