JetStream pull over export service response type stream

[roeschter]

Suggested fix - no unit tests provided yet

When doing a Jetstream fetch request to a stream across an account boundary we export:
{ service: "$JS.API.>", accounts: [ "B" ], response_type: "stream" }

When the reply stream contain a nil message (no body no headers) the import reply subject mapping is removed. This seems to be by design. To Jetstream this looks like a message loss and results in timeouts and retries. Note that not just the nil message is suppressed but all messages coming after the nil messages are not delivered either.

Adding a header to all messages (e.g. Nats-Msg-Id) prevents this effect.

The fix here make removeRespServiceImport() Jetstream aware. Not removing it for messages which which $JS.ACK set. This is a minor fix with no performance impact as the check for JS reply subjects is done anyway further up in the code.

Also see internal Zendesk ticket [986]

Signed-off-by:m Michael Roeschter michael@synadia.com

[derekcollison]

Need tests.

[derekcollison]

Also this trick would not work for ack none IIRC since no reply would be given.

[roeschter]

Added two tests.

1. TestMultiRequestReplyExportImportWithEmptyMessages - Tests that reply subject mapping for regular messages correctly breaks on nil message
2. TestJetStreamFetchAcrossAccountExportImportWithEmptyMessages - Tests that reply subject mapping does NOT break for JS message and therefore fetch works correctly

[roeschter]

Also this trick would not work for ack none IIRC since no reply would be given.

I just tested it an even with AckPolicy None, $JS.ACK is present

[roeschter]

Customer for the time being used the workaround/recommendation to always set a header in Jetstream messages.

[neilalexander]

Is this still relevant?

[neilalexander]

Closing for now, but we can reopen if we need to revisit.