http://web.archive.org/web/20150325003241/http://blog.founda...
I'm not trying to make a comparison between a system I used to work on and one that I frankly know little to nothing about; rather, I'd suggest that building a system like this just isn't enough to be compelling on its own.
As a business it was always an ambitious effort, and I'm not sure what could or should have been done differently. But since then I've used a number of other systems and thought to myself "boy, I wish I had FDB right now."
You're right that distributed consistency is a beginning, not an end. We are painfully aware of all the startups that have died or are dying on this beach.
It's great to be scalable and consistent, but you have to be more than an operationally-better replacement for legacy SQL. That's one reason we built our own query language that plays to modern application development patterns (serverless, functional, change feeds, etc.) instead of the typical slow, never-quite-there, distributed SQL planner.
On mobile so cannot read article
You're doing 3k batches per second with 4 logical writes each, right? So that is at most 3-12k writes per second using the way that every other distributed database benchmark and paper counts.
Or otherwise - if you continue counting writes in this special/misleading way - you'd have to multiply every other distributed db benchmark's performance numbers with a factor of 3-15x to get an apples-to-apples comparison.
The 12k batched writes/sek through what I assume is a paxos variant is still pretty impressive though! Good to get more competition/alternatives for zookeeper & friends!
If you want to include write amplification, then multiply by 6x again to account for the replicated log and the tables themselves.
Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores.
12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it to key value stores. But of course, comparing Fauna with eventually consistent systems is not a really fair comparison. You don't make it fairer by misleading in your benchmark though.
Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.
120,000 writes per second is accurate, talking about actual durable storage (disk) writes. But it's only 3,330 transactions, which should be the number that a user cares about.
I don't have proper data and I'm a bit rusty, but I feel like Cassandra could blow that away if you set similar consistency requirements on the client side (QUORUM on read, same for write?). Am I understanding this correctly, or does Fauna/Calvin give you something functionally better than what C* can do?
YMMV, but we've found the performance of Cassandra writing out similar-sized multi-row atomic batches at QUORUM to be similar in this hardware configuration.
FaunaDB transactions are quite a bit more powerful, as they can span multiple keys, use conditionals and read-modify-write logic, and still resolve with serializable semantics.
The 1M "TPS" you're referring to is a read-only benchmark (e.g. http://akorotkov.github.io/blog/2016/05/09/scalability-towar...). Those are reads (most likely from the buffer cache), not writes or transactions in any real sense.
On the other hand, the Fauna numbers don't seem that impressive to me. On a mid-2011 Macbook Air, I get 2600 transactions per second (read-committed) in PostgreSQL 9.6. Setup is as follows:
CREATE TABLE IF NOT EXISTS foo(a TEXT, b TEXT, c TEXT, d TEXT);
CREATE INDEX IF NOT EXISTS idx_foo_a ON foo(a);
CREATE INDEX IF NOT EXISTS idx_foo_b ON foo(b);
CREATE INDEX IF NOT EXISTS idx_foo_c ON foo(c);
CREATE INDEX IF NOT EXISTS idx_foo_d ON foo(d);
-- prepared statement, the inserted strings are 4 chars wide
INSERT INTO foo(a, b, c, d)
VALUES
($1, $2, $3, $4),
($5, $6, $7, $8),
($9, $10, $11, $12),
($13, $14, $15, $16);
Am I missing something?
If you ran the benchmark yourself, how did you achieve 1m durable writes/sec on a postgres machine/instance? [It's quite an achievement] On what kind of crazy hardware? How large was each write/row? Did you use the postgres network protocol to perform the writes?
How does this algorithm compare to whatever Google Spanner does?
This makes the read performance equivalent to something like Cassandra at CONSISTENCY.ONE, without giving up the cross-partition write linearization of something like Spanner.
I've personally seen a Cassandra ring go to more than 2M ops/sec.
Is this specifically for distributed SQL only? I think there are some scalable SQL systems that don't support sessions either.
Multi-query transactions can be useful, but the FaunaDB query language is functional, rather than declarative like SQL, so composing queries that can do everything you want is usually easier than SQL.
Would you create a single operation that reads one record, checks that it's enough, then adds the amount to another record?
Or maybe you'd first read both accounts, then issue a conditional write operation that makes sure the data hasn't changed before doing the write?
http://techblog.netflix.com/2011/11/benchmarking-cassandra-s...
Also a single SSD from 2015 is rated at 120K writes per second:
PM1725: http://www.samsung.com/semiconductor/global/file/insight/201...