Managing Large-Scale Redis Clusters on K8s – Kuaishou's Approach (opens in new tab)

(kubeblocks.io)

26 pointswest0n1y ago5 comments

5 comments

nkzd1y ago

In companies I've worked at I noticed databases (Both SQL and No-SQL) are never hosted in Kubernetes clusters. I am looking for opinions of more senior folks on why is that? Is Kubernetes simply not mature enough?

kawsper1y ago

For us, our Nomad cluster clients (the instances that runs the workloads), are a bit more ephemeral, the workloads are moved around servers, and the servers go up and down as we scale the infrastructure.

A database, that should be highly available, that is on a node that's going down, might disrupt database clients and end-users.

Not to say that you can't host databases inside the cluster, you can place databases on more stable servers (tag them with "db", be careful when you make changes) and don't move workloads, or manage expectations with the database clients.

sethammons1y ago

Most of my k8s experience is from using it 4 years ago, so it may be better with storage now. But we required bare metal for all our dbs. We required full performance.

We did have redis in k8s, but the redis cluster our team needed was over 50 nodes and was experiencing all kinds of odd problems and our ops team insisted we move to vms. Performance was better. We didn't use disc, all memory. Wish I could recall the issues better.

fernandotakai1y ago

from my own experience, running stateful services in k8s is not easy and prone to errors.

so unless your devops/sysops/whateveryouwannacallitops has a lot of experience, it's just not worth it when things like RDS/ElastiCache exists.

i worked at a place that was 100% k8s, but all stateful things (redis, mysql, rabbitmq) were either hosted or in VMs.

sluongng1y ago

Totally noob question: what is "redis-proxy" in this doc? Envoy with Redis protocol codec?

j / k navigate · click thread line to collapse