by aphyr on 1/31/24, 3:04 PM with 69 comments
by hudo on 1/31/24, 3:53 PM
Interesting trivia: there's "raven db done right" - https://martendb.io/ , just an API wrapper around PSQL. Named Marten because thats a natural enemy of ravens:)
by mjb on 1/31/24, 5:11 PM
- Replication is a technique used to achieve higher availability and durability than a single node can offer, by making multiple copies of the data. Techniques include Paxos, Raft, chain replication, quorum protocols, etc.
- Active-active means that transactions can run against multiple different replicas at the same time, while still achieving the desired level of isolation and consistency.
- Atomic commitment is a technique used in sharded/partitioned databases (which themselves exist to scale throughput or size beyond the capabilities of a single machine) to allow transactions to be atomically (“all or nothing”) committed across multiple shards (and allow one or more shards to vote “nah, let’s not commit this”). 2 phase commit (2PC) is the classic technique.
- Concurrency control is a set of techniques to implement isolation, which is needed in any database that allows concurrent sessions (single node or multi-node). Classic techniques include 2PL and OCC, but many exist.
When vendors or projects answer concurrency control questions with replication answers (which appears to be the case here), it’s worth diving deeper into those answers. There are cases where “Paxos” or “Raft” might be answers to atomic commitment or even concurrency control questions, but at best they are very partial answers and building blocks of a larger protocol. Databases that only support “single shot”/predeclared transactions can get away without a lot of concurrency control, for example, and might be able to do the required work as part of their state machine replication protocol. In general, I'd see using words like "Paxos" and "Raft" in the marketing for a database as a negative sign. It's not a fully reliable one, but it's often the least interesting part of the implementation and the choices the database is making.
To be extra clear, I’m not criticizing Aphyr here (the article clearly doesn’t conflate these concepts), but more pointing out what I think lies at the bottom of a lot of the issues we see with distributed database claims.
by CJefferson on 1/31/24, 3:30 PM
by Twirrim on 1/31/24, 4:03 PM
by gigatexal on 1/31/24, 3:58 PM
by ukd1 on 1/31/24, 4:42 PM
by PreInternet01 on 1/31/24, 6:24 PM
It did not... go exactly as planned. Initial tests looked OK, but when I did testing with actual users, there were huge issues right away. Like: OK, I just ran your ingestion pipeline. What do you see? And the answer was 'well, nothing', or 'ehhm, a lot less than I expected'. These issues turned out to be pretty much impossible to fix: there were no real errors, but the data just seemed to... disappear randomly, even in a simple single-node cluster. I got community support involved in a bunch of particular issues, but nothing really helped: the aggregate numbers we got never added up to what they should be.
I then migrated the whole thing to a single SQLite database. That file is, as I write this, a good 2TB in size, and still performs as well as the day it was deployed and never had any unexplained-number issues, without any changes to the surrounding code. I did eventually move away from the .NET Entity Framework (as that did cause some rare, yet unexpected and hard-to-fix concurrency issues, but those were hard crashes and not silent data corruptions) to a hand-rolled entity mapper, but all has been good since then...
TL;DR: databases are very hard, and fashionable choices are not necessarily desirable.
by philipbjorge on 1/31/24, 7:11 PM
Won't go into battle scars here, but this report does not surprise me. We're much happier with Postgres and Elasticsearch.
by dramm on 1/31/24, 8:39 PM
by RcouF1uZ4gsC on 1/31/24, 3:52 PM
I think in 99.9% of cases, you don't want AP. The P only matters when the network is more prone to go down than the machines. For example, if every node goes down, your AP design won't be available.
With the massive improvements in network and connectivity and increased redundancy, you should aim for CP.
If you really, really need AP, then a ground up design based on CRDTs seems the best, most discipline approach. With CRDT, you can have availability because the operations can be entirely local, and you know you can sync to the other nodes when available without conflict.
by jjirsa on 1/31/24, 4:21 PM
by endisneigh on 1/31/24, 4:13 PM
by neonsunset on 1/31/24, 10:39 PM
But RavenDB does not do it justice and uses unsafe in catastrophic amounts in places where it is not necessary or in ways which are straight up UB despite the fact that JIT/ILC is much more strict than GCC/LLVM. There have been multiple bug reports submitted to dotnet/runtime by RavenDB which required extensive debugging effort only to end up being an issue on RavenDBs end due to explicit misuse of unsafe APIs (in ways, I must reiterate, that have safe alternatives to achieve the same performance).
(if anyone's interested, I can later ask around/dig through issue history and give the references)
by jwr on 1/31/24, 6:45 PM
I witnessed RethinkDB losing to MongoDB in spite of being significantly better. I am now worried that FoundationDB isn't gaining popularity, even though it is arguably the best and most well-tested distributed database out there, with strict serializability (!) guarantees. But it doesn't have a shiny website and doesn't cause warm fuzzies, quite the opposite, it looks complex and intimidating. So it isn't popular.
This is worrying, but perhaps neither new nor surprising: we have a history of picking inferior solutions because the good ones looked too complex or intimidating (betamax vs VHS in video formats, ATM vs Ethernet in WANs).
by JazCE on 1/31/24, 7:14 PM
by CyanLite2 on 1/31/24, 8:21 PM
I mean who posts that kinda stuff on their public website?
by oliverpk on 1/31/24, 9:01 PM
by romanovcode on 1/31/24, 3:40 PM