Possible to keep the queue alive when the process exists?

[drwxmrrs]

Hi, thanks for this amazing package!

Is it possible to keep the queue alive even when the node process exists?

i.e. if the consumer goes down (and thus removes the queue) will this mean that any new messages in the exchange won't be sent to the queue?

[mateodelnorte]

You just need to use servicebus-retry and use { ack: true } as an option on your listen or subscribe calls. Looks for the above in the docs.

…

On Wed, Feb 19, 2020 at 7:25 PM Drew ***@***.***> wrote: Hi, thanks for this amazing package! Is it possible to keep the queue alive even when the node process exists? i.e. if the consumer goes down (and thus removes the queue) will this mean that any new messages in the exchange won't be sent to the queue? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#150?email_source=notifications&email_token=AAEHOXY5CJU4RFBVPYBMH2DRDXLXPA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IO23TPA>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEHOXYCECOKKBUOS55XZDTRDXLXPANCNFSM4KYFC7UA> .

[drwxmrrs]

Ah ok.

So with the retries - is my understanding correct that the retried message is held in memory (or redis) and not put back into the exchange for redelivery?

[mateodelnorte]

A message is not removed from the queue on rabbitmq until it is acknowledged. Redis and memory are simply used for storing correlation ids and associated counts, to determine if a message has been received more than MAX_RETRIES. So, a subscriber connects. The broker sends it the latest message (not deleting from the queue yet). Subscriber processes one or many times, until it calls ack(), and only then does the broker remove the message from the queue. You can find more info in the rabbitmq docs: https://www.rabbitmq.com/consumers.html#acknowledgement-modes

…

On Wed, Feb 19, 2020 at 11:24 PM Drew ***@***.***> wrote: Ah great. So with the retries - is my understanding correct that the retried message is held in memory (or redis) and not put back into the exchange? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#150?email_source=notifications&email_token=AAEHOX3FMUWNMFN4WN74VYLRDYHXFA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMKY7XI#issuecomment-588615645>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEHOX6EOKXRP3LTNYNEJILRDYHXFANCNFSM4KYFC7UA> .

[drwxmrrs]

Ok that makes sense now! I wonder if we could store an attribute on the message itself (eg x-retry-count), ack that message, and republish it back to the exchange? So message comes in: Exception is thrown Middleware will: Capture exception Add a retry-count to the message (at this point 1) Ack this message Push new message with the same attributes back to the exchange (including the retry count) Process continues I’d be happy to test this out and submit a PR - any thoughts on why this may not work? On Fri, 21 Feb 2020 at 3:26 am, Matt Walters <[email protected]> wrote:

…

A message is not removed from the queue on rabbitmq until it is acknowledged. Redis and memory are simply used for storing correlation ids and associated counts, to determine if a message has been received more than MAX_RETRIES. So, a subscriber connects. The broker sends it the latest message (not deleting from the queue yet). Subscriber processes one or many times, until it calls ack(), and only then does the broker remove the message from the queue. You can find more info in the rabbitmq docs: https://www.rabbitmq.com/consumers.html#acknowledgement-modes On Wed, Feb 19, 2020 at 11:24 PM Drew ***@***.***> wrote: > Ah great. > > So with the retries - is my understanding correct that the retried message > is held in memory (or redis) and not put back into the exchange? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #150?email_source=notifications&email_token=AAEHOX3FMUWNMFN4WN74VYLRDYHXFA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMKY7XI#issuecomment-588615645 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAEHOX6EOKXRP3LTNYNEJILRDYHXFANCNFSM4KYFC7UA > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#150?email_source=notifications&email_token=AAFONUCG4JABUZSPIRRKSHDRD2VJNA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMO7PIA#issuecomment-589166496>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFONUHJ4ZIF5RKODLEUQU3RD2VJNANCNFSM4KYFC7UA> .

[mateodelnorte]

If you republish back to the exchange, the message will be queued after any other messages that have arrived. I do think placing the correlation id in the header is a good idea, and that's something I plan to address in the next week.

…

On Thu, Feb 20, 2020 at 2:54 PM Drew ***@***.***> wrote: Ok that makes sense now! I wonder if we could store an attribute on the message itself (eg x-retry-count), ack that message, and republish it back to the exchange? So message comes in: Exception is thrown Middleware will: Capture exception Add a retry-count to the message (at this point 1) Ack this message Push new message with the same attributes back to the exchange (including the retry count) Process continues I’d be happy to test this out and submit a PR - any thoughts on why this may not work? On Fri, 21 Feb 2020 at 3:26 am, Matt Walters ***@***.***> wrote: > A message is not removed from the queue on rabbitmq until it is > acknowledged. Redis and memory are simply used for storing correlation ids > and associated counts, to determine if a message has been received more > than MAX_RETRIES. > > So, a subscriber connects. The broker sends it the latest message (not > deleting from the queue yet). Subscriber processes one or many times, until > it calls ack(), and only then does the broker remove the message from the > queue. > > You can find more info in the rabbitmq docs: > https://www.rabbitmq.com/consumers.html#acknowledgement-modes > > On Wed, Feb 19, 2020 at 11:24 PM Drew ***@***.***> wrote: > > > Ah great. > > > > So with the retries - is my understanding correct that the retried > message > > is held in memory (or redis) and not put back into the exchange? > > > > — > > You are receiving this because you commented. > > Reply to this email directly, view it on GitHub > > < > #150?email_source=notifications&email_token=AAEHOX3FMUWNMFN4WN74VYLRDYHXFA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMKY7XI#issuecomment-588615645 > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAEHOX6EOKXRP3LTNYNEJILRDYHXFANCNFSM4KYFC7UA > > > > . > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > < #150?email_source=notifications&email_token=AAFONUCG4JABUZSPIRRKSHDRD2VJNA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMO7PIA#issuecomment-589166496 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAFONUHJ4ZIF5RKODLEUQU3RD2VJNANCNFSM4KYFC7UA > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#150?email_source=notifications&email_token=AAEHOXZ3JY7NKO5MDYUMYWLRD3UYDA5CNFSM4KYFC7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMQA5DI#issuecomment-589303437>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEHOX5OE3OPP3EWKXCMKX3RD3UYDANCNFSM4KYFC7UA> .

[mateodelnorte]

There may be some other default options we can utilize for retry-count: https://engineering.nanit.com/rabbitmq-retries-the-full-story-ca4cc6c5b493

[mateodelnorte]

Note, all of these repos under @mateodelnorte/bla are slated to be replaced by servicebus specific repos at https://github.com/servicebus. We're undergoing some changes and will release everything under a new major version.