Create a camel 'Broadcaster'

[dannylamb]

Create a small camel route that sends a message to a dynamic list of recipients using the Recipient List design pattern. The list should be read from a header, which you can assume comes with the message. See the 'Dynamic Recipient List' section of this page for an example of this.

Also, see this by @acoburn. It will get you 90% of the way there. The only real difference is it reads from configuration, and we want the value supplied by a header.

[acoburn]

Actually, @bseeger wrote that code :-)

[DiegoPino]

@dannylamb need the message structure/format. Can you elaborate more on where this fits? thanks

[dannylamb]

The message is anything. At least this bundle wouldn't care about actual contents. It will need a header, like say 'X-Islandora-RecipientList', that contains a comma separated list of endpoints to pass the message to.

[acoburn]

No, you don't need the message structure -- this is a simple broadcasting route: it takes a message from a queue A and it sends it to queue B, C and D. The message headers/body is entirely unrelated -- whatever they are will simply be copied to the various endpoints.

[dannylamb]

Precisely. This is just plumbing.

[DiegoPino]

ok, i was thinking about something more fancy.The relay thing is almost(it is) and existing patterns, so does it makes sense to be a single service instead of a route embedded in a more complex service?

[acoburn]

Here is an elaboration with an example:

Fedora produces three messages: M1, M2 and M3
You have four different consumers of these messages: C1, C2, C3 and C4

One option is to publish the messages on a "topic" such that all consumers receive all messages. This is fine until you need to restart one of those consumers, at which point messages may be missed. So you want to use "queue"s instead. But the problem with a queue is that C1 may retrieve M1, C2 may retrieve M2 and C4 may retrieve M3 -- each consumer doesn't retrieve every message and some consumers may not retrieve any messages.

So what you need is a separate queue for each consumer, and that's where this comes in: you will end up with four separate queues and each of the consumers will process messages without missing any. If you look at the code @bseeger wrote, the way it works is you can dynamically define what those queues are (and you can change the configuration at runtime).

[DiegoPino]

So should i look at some of this patterns http://zguide.zeromq.org/hx:all#advanced-request-reply ?
Or just forget it.. I mean the code @bseeger wrote is there and works

Note: zeromq has a camel component https://www.google.com/search?client=safari&rls=en&q=apache+camel+zeromq+component&ie=UTF-8&oe=UTF-8

[dannylamb]

@DiegoPino I imagine the default will be activemq, but there's nothing about @bseeger's code that makes assumptions about what technology you're using. So you could use zeroMQ if you wanted, since I see you're reading their docs.

And request reply is one of my upcoming tickets. We'll use that to create a pipeline, which will be great for letting users control derivative generation. Camel turns that into a one liner : D You just set the exchange pattern to inOut.

Check this out: https://gist.github.com/dannylamb/0e5b92285960561f09c0652b9a925e4e

This broadcaster and that pipeline will be the two main entry points into our asynchronous processing.

[dannylamb]

@DiegoPino If you wanted to use the zeromq camel component, just make a simple bundle that uses blueprint to register the component with a jndi name of 'acrepo/Broker'. Then, so long as your routes look like zeromq:queue:blahblah, then it should all work out for you.

Currently, we're using that technique with activemq here: https://gitlab.amherst.edu/acdc/repository-extension-services/blob/master/acrepo-services-activemq/src/main/resources/OSGI-INF/blueprint/blueprint.xml

[dannylamb]

Which is also @acoburn / @bseeger code.

[dannylamb]

See #449

[dannylamb]

Blocked by #458