KeyDB CEO Interview: Getting into YC with a Fork of Redis (opens in new tab)

100% this.

And to your point about client libraries: for any shared behavior that needs to be local to the app (eg. session handling, permissions, etc.), it's more appropriate to spin up a sidecar that you can talk to over sockets than to try to build client libraries in each and every language. Your client libraries will differ in behavior and it is an incredible pain keeping them all up to date, patching every app, etc.

Envoy for S2S and traffic mesh + sidecars for shared behavior is better than building client smarts.

But the Envoy proxy only supports a subset of the commands. For example, it doesn’t support LPOPRPUSH, precisely because the keys could be stored in multiple nodes, and they don’t support that part of the clustering protocols.

Really Cool project. Just a quick thing, can you make the html responsive with the viewport meta tag?

Yes. KeyDB is legit, it works well, it's free speed in a lot of cases, the maintainer is a nice guy and is very responsive. We used it in production and it had the same robustness you'd expect from Redis.

I recommend keydb in almost every case redis is used.

Yea FLASH is only in the Enterprise version. At a rough level the open source is the best cache we can make while pro/enterprise is the best full featured database. It’s not fully there yet but the end goal is you can get caching and persistence in one place.

Someone on my team looked at it, and for our workload (lots of MGETs per query) it wasn't that much better than redis. We may still go with it though, our GCP Memorystore instance is pretty much at 100% CPU constantly.

We have still yet to evaluate Redis6.

Draftbit uses KeyDB in production. It's fast

Take the scenario where you have an 9 node redis cluster. With KeyDB you could shrink that down to 3 nodes with 3 threads while getting the same throughput. This reduces the maintenance burden while also reducing latency. There's overhead to connecting to a different cluster server each time you access a seperate shard.

Even better you might be able to avoid having a cluster at all. For many that’s the biggest win with KeyDB.

This what redis’s creator antirez suggests: “ the way I want to scale Redis is by improving the support for multiple Redis instances to be executed in the same host, especially via Redis Cluster” http://antirez.com/news/126

It’s basically back to the multi-threading vs multiprocessing debate that also exists in Python. With multiprocessing, you have more overhead but the client itself is simpler.

It’s kind of a weird in between where you need more scalability then a single redis but no so much that you want to go to a cluster.

They have Elasticache which is always a concern. In terms of open source projects my read is they really don’t want to run their own and are much more comfortable operating open source as a service. They “forked” Elastic Search 2 years ago and basically did nothing with it until it was re-licensed a few months ago. Now that they are investing more into it I’m interested to see how they handle it long term.

Yea that was roughly KeyDB’s original design. There’s some nuance in the locking like ensuring it’s both fair and fast so P99 latency doesn’t get out of control.

In the Enterprise codebase we can take snapshots which lets us do reads without the lock but it’s a bit of work to enable for commands so it only applies to KEYS and SCAN at the moment.

I’ve learned to appreciate Salvatore’s stance on simplicity the longer we’ve gone on with KeyDB. But when I first made KeyDB two years ago I was really perplexed at the decisions he made with respect to threading.

It’s not my intention to be antagonistic. I’ve had a lot of projects over the years that went nowhere and a part of me is sad that the one with the most traction is a fork.

I don't read that as antagonistic. I think you're just being too sensitive, or reading a tone into the test that isn't actually there.

We have our FLASH feature which gives us a baseline persistence story, you don't need RDBs or AOFs anymore. However there's a lot of work left to do in this area.

The long term goal of KeyDB is to let you balance your dataset across memory and disk in one database. In the future I think caches will just be a feature of a more full featured database and that's where we're heading with KeyDB.

I don't think more data than fits in memory is a requirement. An important innovation in databases has been the realization that there are important performance benefits from assuming all data can fit in memory even if you still provide persistence.

Even persistence isn't really required. A pure analytics view of another database is still a database by itself, but it doesn't need to actually persist anything. It seems like querying is more important to the concept of a database rather than actual storage.

There are a few, some predate us like Threadis. Most only offer a subset of the Redis feature set where we really wanted to be a drop in replacement.

Here it is the repo https://github.com/ideawu/ssdb

this is slightly different as it is disk based. but damn, how can it be faster than RAM

They’re users. We were pretty bad at asking for logos and just started down that road. Honestly if I had one regret it’s not doing more user outreach.

KeyDB is a few % slower single threaded right now, but we have some cool announcements coming in that regard.

Some people use us for Active Rep or some of our other custom commands rather than just the Multithreading.

MemKeyDB is fork of Redis using both: DRAM and Persistent Memory.

> zero-cost multi-threading

I think you mean zero cost abstractions. Which aren't usually zero cost, but just zero additional cost over doing it yourself.

There's no such thing as zero cost multi threading. Just tradeoffs. Rust actually doesn't help with performance here (it gets in the way often) but it definitely does help with correctness - which is truly hard with multi threaded programs.

> You would get the zero-cost multi-threading,

You kinda have to look at how things really work underneath before you can apply buzzwords to a database.

Tokio async runtime for Rust has a tutorial in its user guide https://tokio.rs/tokio/tutorial on writing a mini-redis (https://github.com/tokio-rs/mini-redis).

I've done a C clean room version and I will say that the networking part is as important as the multi-threading the data structures: https://github.com/raitechnology/raids/.

If you go to the landing page of the above, scroll down to the bottom, there is a TCP bypass solution graphed, using Solarflare Open Onload and it is capable of running several times as fast as the Linux Kernel TCP. I didn't test Redis with Open Onload, but I'm pretty sure you'll get a similar results since TCP is a major performance bottleneck in Redis as well.

It's been a long time since I've looked at KeyDB, but IIRC KeyDB is just Redis plus a spinlock. It's actually still very performant. There are other "toy" reimplementations of Redis in Rust that take the same approach and aren't even as performant as single threaded Redis.

The next approach you could take is using something like Glommio and take a thread-per-core design to Redis. I think that approach has a lot of potential, but the design becomes more complex (you now need something like distributed transactions for "cross-core" Redis transactions and mutli-gets)