by celrenheit on 12/11/17, 8:43 AM with 68 comments
by menacingly on 12/11/17, 3:19 PM
I'm not saying I wouldn't give this project a closer look, but I would much rather a product make it painfully obvious what compromises were made to offer its availability.
My instinct is: either you need strict constraints on ordering and delivery, in which case you use rabbit, or you need at-least-once-no-matter-what semantics, in which case you use something else and make your app less fragile.
by manigandham on 12/11/17, 12:09 PM
It's worth mentioning the new Apache Pulsar messaging system which can replace Kafka with pub/sub and queueing semantics while providing better scalability and per-message acks, probably better suited to those who want a combined system.
by adrinavarro on 12/11/17, 10:07 AM
I'm currently dealing with a queuing-related issue.
I have a series of tasks running across servers that consume from a queue and run a task.
Often, these tasks die mid-execution (but can be resumed by any other server). So, the queue is a database, and the running tasks "touch" a timestamp in the database if they are still executing. When a database document hasn't been updated for a while, the "consumption query" makes it so that it is 'redelivered' to an available server listening to the "queue".
Of course, this is subpar, but we haven't yet come across an elegant (and not too over-engineered) way to replace this.
by eddd on 12/11/17, 11:07 AM
> The first is to be able to track each message individually (i.e. not using a single commit offset) to make suitable for asynchronous tasks. > The second is the ability to schedule messages to be consumed in the future. This make it suitable for retries.
That's a start - I'd love to see how did you solve the problem? How your solution compares to other similar products? Why should I care about stuff you are mentioning? It is not an issue of not mature product - I think one should start with defining exactly and precisely what is being solved here.
When solving a technical problem you always have to tailor your solution to a specific set of requirements and it is never like: "distributed, horizontally scalable, persistent, time sorted message queue." So please stop using such buzzwords.
by jwr on 12/11/17, 1:05 PM
by agnivade on 12/11/17, 12:12 PM
by nicois on 12/11/17, 10:54 AM
by dvdplm on 12/11/17, 10:06 AM
by tmp123tmp123 on 12/11/17, 1:06 PM
by lloydatkinson on 12/11/17, 11:43 AM
by velodrome on 12/11/17, 10:53 AM
by ninjamayo on 12/11/17, 9:22 AM
by yassinebenyahia on 12/11/17, 9:01 AM