Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.
It's an infinitely more friendly version of Kafka, pub/sub, etc that is extremely lightweight. Yet every environment trends towards Kafka because it was "chosen" by the big corps.
It's nothing to do with these apps being superior in any way - often it is the opposite - like reasonable faults and glitches can add more hours to bill the customers for "fixing".
"Key value", "pub-sub", "microservices can live anywhere" And?
Why do I care? What does "microservices can live anywhere" even mean and what does it have to do with NATS?
I find RabbitMQ sometimes inscrutable but I think even their website is better
The new Khepri metadata store in RabbitMQ 4.0 allows for even better scalability.
Just wondering what the biggest selling points are for somebody making a decision?
We needed features like job priorities, and complex routing logic and on top of that our software needs to send millions of messages a minute. What we miss is a nice UI to see the state of jobs, and monitor for problems more easily like: https://taskforce.sh/
> RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine.
What does that mean? "Messaging and streaming broker"? I understand the need for worker queues to process videos, images, emails and such but I can't easily tell if that's what this is.
Also, what are the benefits of this over just processing incomplete records straight out of my database? i.e. using MySQL as a queue.
Mainly throughput and latency. I haven’t used MySQL recently so some of this may apply more to Postgres.
Postgres has LISTEN/NOTIFY which helps with latency. I don’t think MySQL has LISTEN/NOTIFY, which means you’d have to resort to polling.
You have to use the `SELECT … FOR UPDATE SKIP LOCKED LIMIT 1` features to grab a message from the queue table, so multiple consumers don’t pull the same message.
The biggest issue, if you’re trying to achieve decent throughput, is dealing with bloat (empty pages still sitting on the disk that haven’t been cleaned up yet). You can run vacuum but an online vacuum will only mark the pages as available for reuse (doesn’t free up the disk space). And if you run a full vacuum (which will free the disk space) it locks the entire database while it runs the vacuum. This can compound if you’re using indexes.
One way of dealing with this is setting up partitioning by message timestamp, so that as old messages roll out, you can just drop those partitions and not have to deal with vacuum.
It can work if your queue needs are low throughput or can tolerate higher latency, but there are some things to manage, and realistically setting up a Redis instance is probably less complex than trying to work around the database-specific quirks, unless you’re already very familiar with your database’s inner workings.
If you are looking at RabbitMQ with "Maybe this is too much". Beanstalkd likely has features you need with almost none of the setup. Just don't web expose it ;)
I only used it on one project years ago and it was a pleasure, dead easy to get up and running and rock solid.
You have to read a lot of docs or you WILL hold RabbitMQ wrong.
The ideal RMQ cluster has 3 servers and is dedicated to just a few apps.
The biggest issue back then was finding a quality client implementation for the language you were using. Not sure what the status of that is these days.
Its unfortunate your team ran into performance issues, as Erlang can be inefficient in some situations. Were you using static routes on the DNS load balanced parallel consumers, or relying on the disk caching mechanisms?
I haven't touched it in years so I can't expand, but when I did, I had to write so many wrappers and add extra logic to use it properly.
Does RabbitMQ have equivalent features?
Reminds me, I'll have to check if they have a working donation link someplace. =3
BEAM languages (including Elixir and Gleam) share the benefits Erlang enjoys by also being part of the ecosystem.
I loved it and the only issues we had were due to our fuckups.
Perhaps people are still thinking in single point of ingress design paradigms. Admittedly RabbitMQ can be a pain to administer for Juniors, but then again so are the other options. =3
I can’t speak to the new version, but it comes with support for even more messaging patterns out of the box.
Earlier this year I tried setting it up as a websockets message broker in a Spring Boot app but failed miserably. I ended up using Spring’s simple broker.
Edit: FWIW, NATS also supports MQTT: https://docs.nats.io/running-a-nats-service/configuration/mq...
They didn't lose my respect for the removal of the argument, however, they lost my respect for whatabouting the conversation calling SemVer a "no true scotsman" fallacy, then trying to claim that removing a redundant argument is not a breaking change, and other reality-warping nonsense, before blocking myself and other complainants from their issues - and even deleting some of their own comments to mop up some of their own terrible reasoning.
I'm sure there is no love lost on their side, either. Personal rant over.
Well, at least AMQP 1.0 is now supported so I expect that for most things you are able to use any client now.
https://docs.celeryq.dev/en/stable/getting-started/backends-...
https://github.com/rabbitmq/rabbitmq-server/blob/main/releas...