Missing return values from fan out / fan in with retry #318
Description
🐛 Describe the bug
If an action functions fails (but not enough times to exceed settings of the retry options) the returned value is
None
Note: It may very well be that I am doing something wrong, but I have not been able to figure out what.
🤔 Expected behavior
If the action function succeeded after retry, then the actual value should be returned, else the action call should raise an exception when
yield context.task_all(...)is called.
☕ Steps to reproduce
Created a simple repro sample here: https://github.com/Vidarls/durable_stutter
But it is short, so I will also add the code here for reference:
(It is a simulated fan out / fan in IO sample, simulating intermittent failure in the action function
with varying execution time)
# Starter
import logging
import azure.functions as func
import azure.durable_functions as df
async def main(req: func.HttpRequest, starter: str) -> func.HttpResponse:
client = df.DurableOrchestrationClient(starter)
instance_id = await client.start_new('execute', None, None)
logging.info(f"Started orchestration with ID = '{instance_id}'.")
return client.create_check_status_response(req, instance_id)
# orchestrator
import azure.functions as func
import azure.durable_functions as df
from azure.durable_functions.models.RetryOptions import RetryOptions
def orchestrator_function(context: df.DurableOrchestrationContext):
retry_policy = RetryOptions(
first_retry_interval_in_milliseconds=1500,
max_number_of_attempts=3
)
tasks = [context.call_activity_with_retry('do', retry_options=retry_policy, input_=i) for i in range(100)]
results = yield context.task_all(tasks)
sorted_items = sorted(i for i in results if i is not None)
missing_items = set(range(100)).difference(sorted_items)
return {
'sorted_len': len(sorted_items),
'len_all': len(results),
'missing_items': list(missing_items),
'sorted_items': sorted_items,
'items': results
}
main = df.Orchestrator.create(orchestrator_function)
# Action function
import random
import asyncio
random.seed('A random sentence for seed')
options = [True, False]
odds = [5, 95]
def maybe_fail():
fail = random.choices(options, odds)[0]
if fail:
raise Exception('Chaos monkey do bad')
async def main(i: int) -> str:
duration = random.randint(1,10)
await asyncio.sleep(duration)
maybe_fail()
return iSample response
(leaving out the long lists for brevity)
{
"sorted_len": 97,
"len_all": 100,
"missing_items": [
16,
10,
79
],
}⚡If deployed to Azure
n/a