FIX recursive_will_execute performance (simple ~300x performance increase}

[ricklove]

This solves a major performance problem that makes large graphs impossible to use in ComfyUI. It also speeds up even medium size graphs that have a long chain of dependent nodes.

Performance improvement example:

• sorting nodes len(to_execute): 40
• recursive_will_execute Execution time: 46.01077870000154 secs
• recursive_will_execute_len Execution time: 0.1593280000379309 secs

Measurement code (in execution.py):

                #always execute the output that depends on the least amount of unexecuted nodes first
                print(f'sorting nodes len(to_execute):{len(to_execute)}')

                start_time = time.perf_counter()
                to_execute_old = sorted(list(map(lambda a: (len(recursive_will_execute(prompt, self.outputs, a[-1])), a[-1]), to_execute)))
                end_time = time.perf_counter()
                execution_time = end_time - start_time
                print(f'recursive_will_execute Execution time: {execution_time} secs')

                start_time = time.perf_counter()
                memo = {}
                to_execute_old = sorted(list(map(lambda a: (len(recursive_will_execute_memo(prompt, self.outputs, a[-1], memo)), a[-1]), to_execute)))
                end_time = time.perf_counter()
                execution_time = end_time - start_time
                print(f'recursive_will_execute_memo Execution time: {execution_time} secs')

                start_time = time.perf_counter()
                memo = {}
                to_execute = sorted(list(map(lambda a: ((recursive_will_execute_len(prompt, self.outputs, a[-1], memo)), a[-1]), to_execute)))
                end_time = time.perf_counter()
                execution_time = end_time - start_time
                print(f'recursive_will_execute_len Execution time: {execution_time} secs')

functions:

def recursive_will_execute(prompt, outputs, current_item):
    unique_id = current_item
    # print(f'recursive_will_execute {unique_id}')

    inputs = prompt[unique_id]['inputs']
    will_execute = []
    if unique_id in outputs:
        return []

    for x in inputs:
        input_data = inputs[x]
        if isinstance(input_data, list):
            input_unique_id = input_data[0]
            output_index = input_data[1]
            if input_unique_id not in outputs:
                will_execute += recursive_will_execute(prompt, outputs, input_unique_id)

    return will_execute + [unique_id]

def recursive_will_execute_memo(prompt, outputs, current_item, memo):
    unique_id = current_item
    # print(f'recursive_will_execute_memo {unique_id}')

    if unique_id in memo:
        return memo[unique_id]

    inputs = prompt[unique_id]['inputs']
    will_execute = []
    if unique_id in outputs:
        return []

    for x in inputs:
        input_data = inputs[x]
        if isinstance(input_data, list):
            input_unique_id = input_data[0]
            output_index = input_data[1]
            if input_unique_id not in outputs:
                will_execute += recursive_will_execute_memo(prompt, outputs, input_unique_id, memo)

    memo[unique_id] = will_execute + [unique_id]
    return memo[unique_id]

def recursive_will_execute_len(prompt, outputs, current_item, memo):
    unique_id = current_item
    # print(f'recursive_will_execute_len {unique_id}')
    
    if unique_id in memo:
        return memo[unique_id]

    inputs = prompt[unique_id]['inputs']
    will_execute = 0
    if unique_id in outputs:
        return 0

    for x in inputs:
        input_data = inputs[x]
        if isinstance(input_data, list):
            input_unique_id = input_data[0]
            output_index = input_data[1]
            if input_unique_id not in outputs:
                will_execute += recursive_will_execute_len(prompt, outputs, input_unique_id, memo)

    memo[unique_id] = will_execute + 1
    return memo[unique_id]

[ricklove]

#2666 related

[ricklove]

Here is a simpler version for quick patching after an update:

def recursive_will_execute(prompt, outputs, current_item, memo={}):
    unique_id = current_item

    if unique_id in memo:
        return memo[unique_id]

    inputs = prompt[unique_id]['inputs']
    will_execute = []
    if unique_id in outputs:
        return []

    for x in inputs:
        input_data = inputs[x]
        if isinstance(input_data, list):
            input_unique_id = input_data[0]
            output_index = input_data[1]
            if input_unique_id not in outputs:
                will_execute += recursive_will_execute(prompt, outputs, input_unique_id, memo)

    memo[unique_id] = will_execute + [unique_id]
    return memo[unique_id]

Edit: this also needs a memo={} created where it is called (for best performance), but this PR is now using this

[ricklove]

I switched to the version that has minimal code changes so that any external calls to this function will still work as expected

[ricklove]

notes

[ricklove]

This should be backwards compatible with any external callers of this function.

[ricklove]

memo should be created outside the lambda so it is reused for entire sorting algorithm

[guill]

@ricklove Could you share the workflow and/or code you've used to perf test this? I believe I addressed the combinatorial explosion in #2666 as well (in a more invasive way which likely makes this PR worth merging in the meantime), but I'd like to make sure I'm comparing apples to apples.

[ricklove]

@ricklove Could you share the workflow and/or code you've used to perf test this? I believe I addressed the combinatorial explosion in #2666 as well (in a more invasive way which likely makes this PR worth merging in the meantime), but I'd like to make sure I'm comparing apples to apples.

Sure! I'll create a minimal workflow that uses only basic nodes and has this problem.

btw: I'm excited to see your PR, nested workflows/components are the killer feature (we just have to solve performance)

[ricklove]

This is a workflow that demonstrates the problem. You can also easy change the complexity level with ctrl+b to make chunks of the graph passthrough.

performance-sort-complexity-01.json

[ricklove]

This is a workflow that demonstrates the problem. You can also easy change the complexity level with ctrl+b to make chunks of the graph passthrough.

performance-sort-complexity-01.json

@guill

[rgthree]

It's nice to see this [finally] getting some attention.

This PR doesn't quite solve the entire issues here, and still suffers from millions of wasted cycles and hundreds of seconds--especially on re-execution--for complex workflows. The PR I sent months ago does, though is a bite more complex; perhaps it need not be (#1503 for issue #1502).

Without combing through the differences in the memoization itself, the broad differences are:

1. Apply the same to recursive_output_delete_if_changed
2. Move the memo init above the while, since the prompt doesn't change at all.

@ricklove Do you think you could do this (since, outside PRs seem to fall on deaf ears). Tens of thousands of folks have already using my PR for months, as it's integrated from the outside in rgthree-comfy. I don't know if you intended/checked that it didn't break all those workflows, but it didn't, so thanks for that :)