from Hacker News

Databases at 14.4Mhz

by chton on 12/10/14, 2:53 PM with 81 comments

• by ChuckMcM on 12/10/14, 7:55 PM

  This is Foundation DB's announcement they are doing full ACID databases with a 14.4M writes per second capability. That is insanely fast in the data base world. Running in AWS with 32 c3.8xlarge configured machines. So basically NSA level data base capability for $150/hr. But perhaps more interesting is that those same machines on the open market are about $225,000. That's two rack, one switch and a transaction rate that lets you watch every purchase made at every Walmart store in the US, in real time. That is assuming the stats are correct[1], and it wouldn't even be sweating (14M customers a day vs 14M transactions per second). Insanely fast.

  I wish I was an investor in them.

  [1] http://www.statisticbrain.com/wal-mart-company-statistics/

• by hendzen on 12/10/14, 7:09 PM

  This is very impressive, however...

  See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809

  (All credit for the idea in this comment is due to @aphyr)

  Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throughput. Would be interesting to see it with a Zipfian distribution (or even better, a Biebermark [0])

  [0] - http://smalldatum.blogspot.co.il/2014/04/biebermarks.html

• by jrallison on 12/10/14, 6:06 PM

  We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us.

  We continue to use it for more and more data access patterns which require strong consistency guarantees.

  We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling.

  Excited about this release! My only regret is I didn't find it sooner :)

• by bsaul on 12/10/14, 7:34 PM

  Just watched the linked presentation about "flow" here : https://foundationdb.com/videos/testing-distributed-systems-...

  Is it really the first Distributed DB project to have built a simulator ?

  Because frankly, if that's the case, it seems revolutionary to me. Intuitively, it seems like bringing the same kind of quality improvement as unit testing did to regular software development.

  PS : i should add that this talk is one of the best i've seen this year. The guy is extremely smart, passionate, and clear. (i just loved the The Hurst exponent part).

• by dchichkov on 12/10/14, 7:34 PM

  I'm only familiar with other key-value storage engines, not FoundationDB, but it seems like the goals are: "distributed key-value database, read latencies below 500 microseconds, ACID, scalability".

  I remember evaluating a few low latency key-value storage solutions, and one of these was Stanford's RAMCloud, which is supposed to give 4-5 microseconds reads, 15 microseconds writes, scale up to 10,000 boxes and provide data durability. https://ramcloud.atlassian.net/wiki/display/RAM/RAMCloud Seems like, that would be "Databases at 2000Mhz".

  I've actually studied the code that was handling the network and it had been written pretty nicely, and as far as I know, it should work both over 10Gbe and Infiniband with similar latencies. And I'm not at all surprised, they could get pretty clean looking 4-5us latency distribution, with the code like that.

  How does it compare with FoundationDB? Is it completely different technology?

• by felixgallo on 12/10/14, 6:44 PM

  This looks very interesting and congratulations to the FoundationDB crew on some pretty amazing performance numbers.

  One of the links leads to an interesting C++ actor preprocessor called 'Flow'. In that table, it lists the performance result of sending a message around a ring for a certain number of processes and a certain number of messages, in which Flow appears to be fastest with 0.075 sec in the case of N=1000 and M=1000, compared with, e.g. erlang @ 1.09 seconds.

  My curiosity was piqued, so I threw together a quick microbenchmark in erlang. On a moderately loaded 2013 macbook air (2-core i7) and erlang 17.1, with 1000 iterations of M=1000 and N=1000, it averaged 34 microseconds per run, which compares pretty favorably with Flow's claimed 75000 microseconds. The Flow paper appears to maybe be from 2010, so it would be interesting to know how it's doing in 2014.

• by shortstuffsushi on 12/10/14, 7:39 PM

  As someone who has no idea about the cost of high-scale computing like this, is $150/hr reasonable? It seems like an amount that's hard to sustain to me, but I have no idea if that's a steady, all the time rate, or a burst rate, or what. Or if it's a set up you'd actually ever even need -- seems like from the examples they mention (like the Tweets), they're above the need by a fair amount. Anyone else in this sort of situation care to chip in on that?

• by w8rbt on 12/10/14, 10:27 PM

  I thought it was DB connections over radio waves just above 20 meters. Also, it's MHz, not Mhz.

• by maliki on 12/12/14, 2:10 PM

  This sounds great compared to my anecdotal experience with DB write performance; but is there a collection of database performance benchmarks that this can be easily compared to?

  The best source for DB benchmarking I know of is http://www.tpc.org/. The methodology is more complicated there, but the top results are around 8 million transactions per minute on $5 million systems. This FoundationDB result is more like 900 million transactions per minute on a system that costs $1.5 million a year to rent (so, approx $5 million to buy?).

  The USD/transactions-per-minute metric is clear, but without a standard test suite (schema, queries, client count, etc.), comparing claims of database performance makes my head hurt.

• by illumen on 12/11/14, 5:43 PM

  Impressive.

  However I think there's still plenty of room to grow.

  320,000 concurrent sessions isn't that much by modern standards. You can get 12 million concurrent connections on one linux machine, and push 1gigabit of data.

  Also, 167 megabytes per second (116B * 14.4 million) is not pushing the limits of what one machine can do. I've been able to process 680 megabytes per second of data into a custom video database, plus write it to disk on one 2010 machine. That's doing heavy processing at the same time on the video with plenty of CPU to spare.

  PCIe over fibre can do many transactions messages per second. You can fit 2TB memory machines in 1U (and more).

  Since this is a memory + eventually dump to disk database, I think there is still a lot of room to grow.

• by mariusz79 on 12/10/14, 6:33 PM

  MHz not Mhz.

• by tuyguntn on 12/11/14, 7:24 AM

  I wish I would work with such great professionals as a Junior for 10years, right after school!!!

• by oconnor663 on 12/11/14, 6:06 PM

  Any chance of an open source release for the database core? :)

• by lttlrck on 12/10/14, 5:49 PM

  "Or, as I like to say, 14.4Mhz."

  Sorry, I don't like that at all.

• by imanaccount247 on 12/10/14, 8:34 PM

  Why the deliberately misleading comparisons? If you are doing something genuinely impressive, then you should be able to be honest about it and have it still seem impressive. One tweet is not one write. Comparing tweets per second to writes per second is complete nonsense. How many writes a tweet causes depends on how many followers the person who is tweeting has. The 100 writes per second nonsense is even worse. Do you just think nobody is old enough to have used a database 15 years ago? 10,000 writes per second was no big deal on of the shelf hardware of the day, nevermind on an actual server.