by chuckhend on 5/9/24, 12:21 PM with 109 comments
by bastawhiz on 5/9/24, 3:00 PM
That's not going to be true. It might be true when things are running well, but when it fails, it'll either be at most once or at least once. You don't build for the steady state, you build against the failure mode. That's an important deciding factor in whether you choose a system: you can accept duplicates gracefully or you can accept some amount of data loss.
Without reviewing all of the code, it's not possible to say what this actually is, but since it seems like it's up to the implementor to set up replication, I suspect this is an at-most-once queue (if the client receives a response before the server has replicated the data and the server is destroyed, the data is lost). But depending on the diligence of the developer, it could be that this provides no real guarantees (0-N deliveries).
by pyuser583 on 5/9/24, 3:04 PM
My experience is Postgres queuing makes sense you must extract or persist in the same Postgres instance.
Otherwise, there’s no advantage over standard MQ systems.
Is there something I don’t know.
by ltbarcly3 on 5/9/24, 4:03 PM
https://wiki.postgresql.org/wiki/PGQ_Tutorial https://github.com/florentx/pgqueue https://github.com/cirello-io/pgqueue
Hundreds of them! We have a home grown one called PGQ at work here also.
It's a good idea and easy to implement, but still valuable to have implemented already. Cool project.
by RedShift1 on 5/9/24, 7:14 PM
by conroy on 5/9/24, 2:50 PM
by kelnos on 5/10/24, 5:37 AM
Does Postgres have some features that make it not entirely unsuitable to use for queuing?
by dostoevsky013 on 5/9/24, 5:05 PM
Or do you expect to a dedicated database to be maintained for this queue? Worth comparing it with other queue systems that persist messages and can help you to scale message processing like kafka with topic partitions.
Found this article on how Revolut uses Postgres for events processing: https://medium.com/revolut/recording-more-events-but-where-w...
by anentropic on 5/10/24, 8:58 AM
Looking at https://github.com/tembo-io/pgmq/tree/main/tembo-pgmq-python ...how do I integrate the queue ops with my other db access code?
Or is it better not to use the client lib in that scenario and use the SQL functions directly? https://github.com/tembo-io/pgmq?tab=readme-ov-file#send-two...
by cynicalsecurity on 5/9/24, 6:45 PM
by lloydatkinson on 5/10/24, 9:01 AM
by pulkitsh1234 on 5/10/24, 10:09 AM
by thangngoc89 on 5/9/24, 3:00 PM
by deepsun on 5/10/24, 3:53 AM
I'm yet to find a use case for "WITH TIME ZONE", in all cases it's better to use "WITHOUT TIME ZONE". All it does is displays the date in sql client local timezone, which should never matter for well done service. Would be glad to learn otherwise.
by bdcravens on 5/9/24, 5:58 PM
by rco8786 on 5/9/24, 3:27 PM
by justinclift on 5/10/24, 12:56 AM
by andrewstuart on 5/9/24, 7:45 PM
HTTP is really the perfect client agnostic super simple way to interface with a message queue.
by airocker on 5/9/24, 5:23 PM
by ComputerGuru on 5/9/24, 9:54 PM
by junail on 5/10/24, 3:04 PM
by jilles on 5/9/24, 11:02 PM
by seveibar on 5/9/24, 3:11 PM
The behavior of this library is a bit different and in some ways a bit lower level. If you are using something like this, expect to get very intimate with it as you scale- a lot of times your custom workload would really benefit from a custom index and it's handy to understand how the underlying system works.