Add configurable delay between external task execution

[sreagleeye]

Within the DefaultCamundaWorker, once all the tasks have completed, they are immediately run again. See the code below:

while (!cancellationToken.IsCancellationRequested)
{
    var externalTasks = await SelectExternalTasks(cancellationToken);

    var activeAsyncTasks = externalTasks
        .Select(CreateContext)
        .Select(ExecuteInContext)
        .ToList();

    await Task.WhenAll(activeAsyncTasks);
}

When there are no tasks in any of the topics, this while loop iterates very quickly. On each iteration, the code calls out to POST /external-task/fetchAndLock. This can cause a very high volume of requests to Camunda when there are many workers.

I have not faced any challenges yet in regards to this. However, I do not need my workers to be responding so quickly to tasks. It would be ideal if I was able to set a delay on the worker so that I can reduce the overall load on my Camunda server.

[TechnoBerry]

There are two solutions of your problem:

1. You can configure AsyncResponseTimeout for all DefaultCamundaWorker instances
2. You can provide your own implementation of IExternalTaskSelector or extend existing ExternalTaskSelector and override ExternalTaskSelector#PerformSelection. Your implementation of IExternalTaskSelector can be registered by using ICamundaWorkerBuilder#AddTaskSelector

[sreagleeye]

1. I am already using the AsyncResponseTimeout. This does not help because the request to Camunda comes back within the timeout. Since the response contains 0 results, no task handlers are called, and it sends the fetchAndLock request again.

2. It does look like implementing a new IExternalTaskSelector could work. However, it is not the approach I would have thought of. I don't really want a new way of selecting tasks. I want to do something based on the result of Task Selections.

Perhaps a better feature than my original proposal would be a pre/post handler surrounding the individual task execution. This could also be a place to share logic that must exist across all of your TaskHandlers

while (!cancellationToken.IsCancellationRequested)
{
    var externalTasks = await SelectExternalTasks(cancellationToken);

    // Call pre-execute handler

    var activeAsyncTasks = externalTasks
    .Select(_contextFactory.MakeContext)
    .Select(ExecuteInContext)
    .ToList();

    await Task.WhenAll(activeAsyncTasks);

    // Call post-execute handler
}