from Hacker News

Ask HN: What is the SQLite of nosql databases?

by rhim on 6/13/21, 6:47 AM with 48 comments

I am looking for a simple, file-based nosql database. So basically the sqlite of nosql databases.

  • by ludocode on 6/13/21, 7:30 AM

    As others have mentioned you have lots of options: LMDB, LevelDB/RocksDB, BerkeleyDB. For what it's worth, I spent a long time looking for an embedded key-value store for my current native project since I didn't need the full complexity of SQL. In the end I chose... SQLite.

    All of these embedded NoSQL databases seem to be missing critical features. One such feature for my use case is database compaction. Last I checked, an LMDB database file can never shrink. Full compaction of LevelDB is slow and complicated (as I understand it essentially breaks the levels optimization which is the whole point of the thing.) SQLite meanwhile supports fast incremental vacuum, and it can be triggered manually or automatically.

    SQLite just has everything. Plus the reliability is unmatched. Even if you just need a single table that maps blob keys to blob values, I would still recommend SQLite over any NoSQL database today.

  • by creshal on 6/13/21, 7:27 AM

    The SQLite of NoSQL is still SQLite: https://www.sqlite.org/json1.html

  • by andrey_utkin on 6/13/21, 7:27 AM

    Filesystem.

    Linux has VFS cache which is very robust and efficient.

    Remember that it is typical for programs to read /etc/nsswitch.conf, /etc/resolve.conf and tons of others at startup time - the filesystem is the datasource in Unix tradition, so the machinery is very well optimized.

  • by quantumofalpha on 6/13/21, 7:05 AM

    Isn't the main feature of nosql supposed to be easy horizontal scalability, the exact opposite of storing everything in a single file?

    If you just need a r/w store for some jsons in a single file, why not sqlite? You can put arbitrary-length blobs into it. Some sql will be involved but you can hide it in a wrapper class tailored to your application with a few dozen lines of code or so.

  • by Rochus on 6/13/21, 9:06 AM

    SQLite has a backend which is well suited as a key-value store.

    Here is a NoSql database based on the SQLite backend: https://github.com/rochus-keller/Udb.

    I use it in many of my apps, e.g. https://github.com/rochus-keller/CrossLine. It's lean and fast, and supports objects, indices, hierarchical "globals" like ANSI-M and transactions.

  • by fulafel on 6/13/21, 7:51 AM

    NoSQL databases have many different different data models. Eg object, document, graph, and key/value DBs. In a lot of cases you should probably just use something on top of SQLite, but you should say more about your requirements.

    An interesting one I ran into recently is Datalevin, a Datalog DB on top of LMDB for Clojure: https://github.com/juji-io/datalevin

  • by teekay on 6/13/21, 7:40 AM

    One of those I've tried is LiteDB - https://github.com/mbdavid/LiteDB. I liked it.

    It's small yet capable. If you are familiar with MongoDB, you will feel right at home.

    It's great for .NET developers as it's written in C# but since it's Netstandard 1.3 compatible, you can presumably run it under Ubuntu or Mac OS or wherever else the new .NET 5 runtime works. I've got a C# app running on ARM64 the other day - just saying.

    I wrote about my experience playing with LiteDB here - https://tomaskohl.com/code/2020-04-07/trying-out-litedb/. It's not an in-depth look at all, just a few notes from the field, so to speak.

  • by erk__ on 6/13/21, 7:12 AM

    Probably something like LMDB [0] or Tkrzw [1], though nosql is a bit more diverse in a way SQL is not so it is hard to give a clear answer.

    [0]: https://en.m.wikipedia.org/wiki/Lightning_Memory-Mapped_Data... [1]: https://dbmx.net/tkrzw/

  • by adamansky on 6/13/21, 7:14 AM

    Take a look on the ejdb2 https://ejdb.org

  • by maxk42 on 6/13/21, 7:27 AM

    Take a look at UnQLite: https://unqlite.org/

  • by kenOfYugen on 6/13/21, 7:43 AM

    I ended up writing a small wrapper on top of SQLite based on this: https://dgl.cx/2020/06/sqlite-json-support

    With proper concurrency control, it can work very well even for multi process applications.

  • by xiphias2 on 6/13/21, 7:09 AM

    BerkerleyDB is an older, simpler one, LevelDB / RocksDB are more modern, maintained, better for SSD workload

  • by Tpt on 6/13/21, 8:57 AM

    I like sled that is a nice embedded key value store written in Rust: https://sled.rs/

    However, it is still in heavy development and a bit of a moving target even if the developers are currently heading toward stabilization of the file format.

  • by throwaway888abc on 6/13/21, 6:50 AM

    Check LiteStore as bonus you have api included

    https://h3rald.com/litestore/

    https://github.com/h3rald/litestore

  • by 1vuio0pswjnm7 on 6/13/21, 8:07 AM

  • by Blackthorn on 6/13/21, 7:17 AM

    tkrzw is basically the modern dbm / berkeleydb

  • by abrookewood on 6/13/21, 11:38 PM

    I think you want Mongita. It was featured on HN a while ago: "Mongita is to MongoDB as SQLite is to SQL (github.com/scottrogowski)" https://news.ycombinator.com/item?id=26881915

  • by quickthrower2 on 6/13/21, 11:07 PM

    An object reference :-)

    In memory, high performance, no schema. Get the object to journal to disk and you are almost there!

  • by tutlane on 6/16/21, 12:22 AM

    Its same as sqlite. You can get more info in this sqlite tutorial.

    https://www.tutlane.com/tutorial/sqlite

  • by pistoriusp on 6/13/21, 7:37 AM

    Json on the filesystem

  • by Abishek_Muthian on 6/13/21, 7:39 PM

    There are also persistent key-value store like dbm as part of standard library in python and several 3rd party implementations like bitcask for Go.

  • by diehunde on 6/15/21, 5:10 AM

    You mean an embedded NoSQL database? Because all databases whether SQL or NoSQL are file-based with just different formats and structures.

  • by nicodds on 6/13/21, 7:54 AM

    Personally, I like very much leveldb/rocksdb. They're very fast and solid.

  • by dvfjsdhgfv on 6/13/21, 7:20 AM

    My guess is that although you basically describe BerkeleyDB you probably want Redis.

  • by RustyRussell on 6/13/21, 7:40 AM

    I'd nominate TDB (the trivial database). It's small, robust, effective

  • by CRConrad on 6/14/21, 4:45 PM

    Mostly, but actually not totally, kidding: *.INI files.

  • by jf22 on 6/14/21, 12:37 PM

    RavenDb would be what I would use.

  • by amachefe on 6/13/21, 7:36 AM

    For now, see Postgres as one

  • by basiclaser on 6/13/21, 8:24 AM

    Lodb

  • by voodoochilo on 6/13/21, 7:42 AM