We use data services to do "data related things" that make sense to do at a central proxy layer. This may include caching/coalescing/other logic but it doesn't always, it really depends on the particular use case of that data.
For messages, we don't really cache. Coalescing gives us what we want and the hot channel buckets will end up in memory on the database, which is NVMe backed for reads anyway so the performance of a cache wouldn't add much here for this use case.
In other places, where a single user query turns into many database queries and we have to aggregate data, caching is more helpful.
So once a request is sent to the database, every other instance of the same request (e.g. "hey, fetch me all messages from server id 42") is put on hold. Once the initial request gets an answer from the database, that answer is distrubuted to the inital requester and all those which were on hold. Now if someone is late to the party, they will initiate a new request to the database, because the response is not cached.
I really like this, and this a is great article to share on HN :)
Cassandra was still suffering even with the new data services.
Could you guys elaborate more on the data migrator by any chance?
Rust in the Cloud will make more and more sense as companies focus on optimizing operational costs without compromising scalability and quality.
