Files
ComfyUI/comfy_execution/graph.py
Christian Byrne c226dc23a3 Add --force-output-rerun so output nodes write their files again on a cache hit (#17809)
## Problem

A server that keeps one `PromptExecutor` across jobs and deletes output
files after each job (Comfy Cloud uploads, then removes them) answers a
repeated prompt from the cache: `SaveImage` is not run, the cached
`executed` message replays the first run's filename, that file no longer
exists, and the job finishes as a success with no output. Cloud's own
telemetry counted 2,368 such zero-output successes for 273 users in prod
over 7 days at the start of November (not all of them are this cause).

## Change

New opt-in flag `--force-output-rerun` (default off, so nothing changes
for local users):

- On a cache hit, a node with `OUTPUT_NODE = True` is executed again, so
it writes its files again. Its inputs still come from the cache, so
models, latents and decoded images are not recomputed: the cost is one
encode and write per output node.
- API nodes (`API_NODE = True`) are excluded. Some are output nodes, and
re-running one repeats a paid external call.
- Forced nodes are left out of the `execution_cached` message, so
clients are not told a node was cached when it ran.

## Tests

`tests/execution/test_force_output_rerun.py` starts a real server:

- With the flag: run a prompt, delete its saved file, run the identical
prompt again. The save node writes a new file that exists, its upstream
`EmptyImage` stays cached, and a test output node marked `API_NODE`
stays cached with its first output.
- Control, without the flag: the second run replays the first filename
and every node is cached.

Red first: on the commit with only the flag, the flagged test failed
because the second run replayed the first filename
(`force_rerun_00001_.png`), while the no-flag control passed:
https://github.com/Comfy-Org/ComfyUI/actions/runs/44228691563/job/134548112592.
A follow-up commit corrects one assertion (the counter can reuse a name
after the file is deleted, so the test checks the file exists instead),
and the last commit wires in the behaviour.

Tracking: https://linear.app/comfyorg/issue/BE-4802 (cloud enables the
flag after this ships: https://linear.app/comfyorg/issue/BE-4803).

## Review notes

An independent review raised three points:

- **Upstream evicted from the RAM cache.** If memory pressure evicted an
upstream result but kept the output node's entry, the upstream nodes run
again. That already happens without this flag: `ExecutionList.add_node`
schedules every uncached input of an output node whether or not the
output node itself is cached, so the flag adds only the output node's
own write. No change made.
- **Output nodes created by an expanding node that is itself cached**
are not re-run, because the cached parent does not expand again. This
flag covers output nodes in the submitted prompt, which is how the
frontend sends subgraphs (flattened). Known limit, noted here rather
than widened in this PR.
- **The test only used `--cache-classic`.** Fixed: the flagged test now
also runs under the default RAM-pressure cache.

The Cursor review panel then found that a node reading a forced output
node's result could run before the re-run and read the old value. Fixed:
`ExecutionList` takes a `force_rerun` predicate and treats those nodes
as uncached while planning, with a test
(`test_a_node_reading_a_forced_output_node_gets_the_new_value`). The
flags are now read by truthiness, and `execution_cached` leaves out only
forced nodes this run will execute.
2026-11-17 20:27:40 -08:00

397 lines
17 KiB
Python

from typing import Type, Literal
import nodes
import asyncio
import inspect
from comfy_execution.graph_utils import is_link, ExecutionBlocker
from comfy.comfy_types.node_typing import ComfyNodeABC, InputTypeDict, InputTypeOptions
from comfy_api.internal import _ComfyNodeInternal
from comfy_api.latest import _io
# NOTE: ExecutionBlocker code got moved to graph_utils.py to prevent torch being imported too soon during unit tests
ExecutionBlocker = ExecutionBlocker
class DependencyCycleError(Exception):
pass
class NodeInputError(Exception):
pass
class NodeNotFoundError(Exception):
pass
class DynamicPrompt:
def __init__(self, original_prompt):
# The original prompt provided by the user
self.original_prompt = original_prompt
# Runtime replacements for original nodes, without mutating the submitted prompt.
self.node_overrides = {}
# Any extra pieces of the graph created during execution
self.ephemeral_prompt = {}
self.ephemeral_parents = {}
self.ephemeral_display = {}
def get_node(self, node_id):
if node_id in self.node_overrides:
return self.node_overrides[node_id]
if node_id in self.ephemeral_prompt:
return self.ephemeral_prompt[node_id]
if node_id in self.original_prompt:
return self.original_prompt[node_id]
raise NodeNotFoundError(f"Node {node_id} not found")
def override_node(self, node_id, node_info):
if not self.has_node(node_id):
raise NodeNotFoundError(f"Node {node_id} not found")
self.node_overrides[node_id] = node_info
def has_node(self, node_id):
return node_id in self.original_prompt or node_id in self.ephemeral_prompt
def add_ephemeral_node(self, node_id, node_info, parent_id, display_id):
self.ephemeral_prompt[node_id] = node_info
self.ephemeral_parents[node_id] = parent_id
self.ephemeral_display[node_id] = display_id
def get_real_node_id(self, node_id):
while node_id in self.ephemeral_parents:
node_id = self.ephemeral_parents[node_id]
return node_id
def get_parent_node_id(self, node_id):
return self.ephemeral_parents.get(node_id, None)
def get_display_node_id(self, node_id):
while node_id in self.ephemeral_display:
node_id = self.ephemeral_display[node_id]
return node_id
def all_node_ids(self):
return set(self.original_prompt.keys()).union(set(self.ephemeral_prompt.keys()))
def get_original_prompt(self):
return self.original_prompt
def get_input_info(
class_def: Type[ComfyNodeABC],
input_name: str,
valid_inputs: InputTypeDict | None = None
) -> tuple[str, Literal["required", "optional", "hidden"], InputTypeOptions] | tuple[None, None, None]:
"""Get the input type, category, and extra info for a given input name.
Arguments:
class_def: The class definition of the node.
input_name: The name of the input to get info for.
valid_inputs: The valid inputs for the node, or None to use the class_def.INPUT_TYPES().
Returns:
tuple[str, str, dict] | tuple[None, None, None]: The input type, category, and extra info for the input name.
"""
valid_inputs = valid_inputs or class_def.INPUT_TYPES()
input_info = None
input_category = None
if "required" in valid_inputs and input_name in valid_inputs["required"]:
input_category = "required"
input_info = valid_inputs["required"][input_name]
elif "optional" in valid_inputs and input_name in valid_inputs["optional"]:
input_category = "optional"
input_info = valid_inputs["optional"][input_name]
elif "hidden" in valid_inputs and input_name in valid_inputs["hidden"]:
input_category = "hidden"
input_info = valid_inputs["hidden"][input_name]
if input_info is None:
return None, None, None
input_type = input_info[0]
if len(input_info) > 1:
extra_info = input_info[1]
else:
extra_info = {}
# if input_type is a list, it is a Combo defined in outdated format; convert it.
# NOTE: uncomment this when we are confident old format going away won't cause too much trouble.
# if isinstance(input_type, list):
# extra_info["options"] = input_type
# input_type = IO.Combo.io_type
return input_type, input_category, extra_info
class TopologicalSort:
def __init__(self, dynprompt):
self.dynprompt = dynprompt
self.pendingNodes = {}
self.blockCount = {} # Number of nodes this node is directly blocked by
self.blocking = {} # Which nodes are blocked by this node
self.externalBlocks = 0
self.externalBlockResults = {}
self.unblockedEvent = asyncio.Event()
def get_input_info(self, unique_id, input_name):
node = self.dynprompt.get_node(unique_id)
class_def = nodes.NODE_CLASS_MAPPINGS[node["class_type"]]
valid_inputs = class_def.INPUT_TYPES()
if issubclass(class_def, _ComfyNodeInternal):
valid_inputs, _, _ = _io.get_finalized_class_inputs(valid_inputs, node["inputs"])
return get_input_info(class_def, input_name, valid_inputs)
def make_input_strong_link(self, to_node_id, to_input):
inputs = self.dynprompt.get_node(to_node_id)["inputs"]
if to_input not in inputs:
raise NodeInputError(f"Node {to_node_id} says it needs input {to_input}, but there is no input to that node at all")
value = inputs[to_input]
if not is_link(value):
raise NodeInputError(f"Node {to_node_id} says it needs input {to_input}, but that value is a constant")
from_node_id, from_socket = value
self.add_strong_link(from_node_id, from_socket, to_node_id)
def add_strong_link(self, from_node_id, from_socket, to_node_id):
if not self.is_cached(from_node_id):
self.add_node(from_node_id)
if to_node_id not in self.blocking[from_node_id]:
self.blocking[from_node_id][to_node_id] = {}
self.blockCount[to_node_id] += 1
self.blocking[from_node_id][to_node_id][from_socket] = True
def add_node(self, node_unique_id, include_lazy=False, subgraph_nodes=None):
node_ids = [node_unique_id]
links = []
while len(node_ids) > 0:
unique_id = node_ids.pop()
if unique_id in self.pendingNodes:
continue
self.pendingNodes[unique_id] = True
self.blockCount[unique_id] = 0
self.blocking[unique_id] = {}
inputs = self.dynprompt.get_node(unique_id)["inputs"]
for input_name in inputs:
value = inputs[input_name]
if is_link(value):
from_node_id, from_socket = value
if subgraph_nodes is not None and from_node_id not in subgraph_nodes:
continue
_, _, input_info = self.get_input_info(unique_id, input_name)
is_lazy = input_info is not None and "lazy" in input_info and input_info["lazy"]
if (include_lazy or not is_lazy):
if not self.is_cached(from_node_id):
node_ids.append(from_node_id)
links.append((from_node_id, from_socket, unique_id))
for link in links:
self.add_strong_link(*link)
def add_external_block(self, node_id):
assert node_id in self.blockCount, "Can't add external block to a node that isn't pending"
self.externalBlocks += 1
self.blockCount[node_id] += 1
def unblock(value=None):
self.release_external_block(node_id, value)
return unblock
def release_external_block(self, node_id, value=None):
self.externalBlocks -= 1
self.blockCount[node_id] -= 1
if value is not None:
self.externalBlockResults[node_id] = value
self.unblockedEvent.set()
def get_external_block_result(self, node_id):
return self.externalBlockResults.get(node_id)
def is_cached(self, node_id):
return False
def get_ready_nodes(self):
return [node_id for node_id in self.pendingNodes if self.blockCount[node_id] == 0]
def pop_node(self, unique_id):
del self.pendingNodes[unique_id]
for blocked_node_id in self.blocking[unique_id]:
self.blockCount[blocked_node_id] -= 1
del self.blocking[unique_id]
self.externalBlockResults.pop(unique_id, None)
def is_empty(self):
return len(self.pendingNodes) == 0
class ExecutionList(TopologicalSort):
"""
ExecutionList implements a topological dissolve of the graph. After a node is staged for execution,
it can still be returned to the graph after having further dependencies added.
"""
def __init__(self, dynprompt, output_cache, output_link_callback=None, force_rerun=None):
super().__init__(dynprompt)
self.output_cache = output_cache
self.output_link_callback = output_link_callback
# Optional predicate: node ids that must run again even with a cached result.
self.force_rerun = force_rerun
self.staged_node_id = None
self.execution_cache = {}
self.execution_cache_listeners = {}
def is_cached(self, node_id):
if self.force_rerun is not None and self.force_rerun(node_id):
return False
return self.output_cache.get_local(node_id) is not None
def cache_link(self, from_node_id, to_node_id, from_socket=None):
if to_node_id not in self.execution_cache:
self.execution_cache[to_node_id] = {}
value = self.output_cache.get_local(from_node_id)
self.execution_cache[to_node_id][from_node_id] = value
if from_node_id not in self.execution_cache_listeners:
self.execution_cache_listeners[from_node_id] = set()
self.execution_cache_listeners[from_node_id].add((to_node_id, from_socket))
if value is not None and from_socket is not None and self.output_link_callback is not None:
self.output_link_callback(value.outputs[from_socket])
def get_cache(self, from_node_id, to_node_id):
if to_node_id not in self.execution_cache:
return None
value = self.execution_cache[to_node_id].get(from_node_id)
if value is None:
return None
#Write back to the main cache on touch.
self.output_cache.set_local(from_node_id, value)
return value
def cache_update(self, node_id, value):
if node_id in self.execution_cache_listeners:
for to_node_id, from_socket in self.execution_cache_listeners[node_id]:
if to_node_id in self.execution_cache:
self.execution_cache[to_node_id][node_id] = value
if from_socket is not None and self.output_link_callback is not None:
self.output_link_callback(value.outputs[from_socket])
def add_strong_link(self, from_node_id, from_socket, to_node_id):
super().add_strong_link(from_node_id, from_socket, to_node_id)
self.cache_link(from_node_id, to_node_id, from_socket)
def inhibit_nodes(self, node_ids):
"""Remove pending nodes selected by the currently executing control node."""
assert self.staged_node_id is not None, "Nodes may only be inhibited while a control node is staged"
node_ids = set(node_ids).intersection(self.pendingNodes)
assert self.staged_node_id not in node_ids, "A control node cannot inhibit itself"
for from_node_id, blocked_nodes in self.blocking.items():
if from_node_id not in node_ids:
for node_id in node_ids:
blocked_nodes.pop(node_id, None)
for node_id in node_ids:
for blocked_node_id in self.blocking[node_id]:
if blocked_node_id not in node_ids:
self.blockCount[blocked_node_id] -= 1
del self.pendingNodes[node_id]
del self.blockCount[node_id]
del self.blocking[node_id]
self.execution_cache.pop(node_id, None)
self.execution_cache_listeners.pop(node_id, None)
for listeners in self.execution_cache_listeners.values():
listeners.difference_update({listener for listener in listeners if listener[0] in node_ids})
async def stage_node_execution(self):
assert self.staged_node_id is None
if self.is_empty():
return None, None, None
available = self.get_ready_nodes()
while len(available) == 0 and self.externalBlocks > 0:
# Wait for an external block to be released
await self.unblockedEvent.wait()
self.unblockedEvent.clear()
available = self.get_ready_nodes()
if len(available) == 0:
cycled_nodes = self.get_nodes_in_cycle()
# Because cycles composed entirely of static nodes are caught during initial validation,
# we will 'blame' the first node in the cycle that is not a static node.
blamed_node = cycled_nodes[0]
for node_id in cycled_nodes:
display_node_id = self.dynprompt.get_display_node_id(node_id)
if display_node_id != node_id:
blamed_node = display_node_id
break
ex = DependencyCycleError("Dependency cycle detected")
error_details = {
"node_id": blamed_node,
"exception_message": str(ex),
"exception_type": "graph.DependencyCycleError",
"traceback": [],
"current_inputs": []
}
return None, error_details, ex
self.staged_node_id = self.ux_friendly_pick_node(available)
return self.staged_node_id, None, None
def ux_friendly_pick_node(self, node_list):
# If an output node is available, do that first.
# Technically this has no effect on the overall length of execution, but it feels better as a user
# for a PreviewImage to display a result as soon as it can
# Some other heuristics could probably be used here to improve the UX further.
def is_output(node_id):
class_type = self.dynprompt.get_node(node_id)["class_type"]
class_def = nodes.NODE_CLASS_MAPPINGS[class_type]
if hasattr(class_def, 'OUTPUT_NODE') and class_def.OUTPUT_NODE == True:
return True
return False
# If an available node is async, do that first.
# This will execute the asynchronous function earlier, reducing the overall time.
def is_async(node_id):
class_type = self.dynprompt.get_node(node_id)["class_type"]
class_def = nodes.NODE_CLASS_MAPPINGS[class_type]
return inspect.iscoroutinefunction(getattr(class_def, class_def.FUNCTION))
for node_id in node_list:
if is_output(node_id) or is_async(node_id):
return node_id
#This should handle the VAEDecode -> preview case
for node_id in node_list:
for blocked_node_id in self.blocking[node_id]:
if is_output(blocked_node_id):
return node_id
#This should handle the VAELoader -> VAEDecode -> preview case
for node_id in node_list:
for blocked_node_id in self.blocking[node_id]:
for blocked_node_id1 in self.blocking[blocked_node_id]:
if is_output(blocked_node_id1):
return node_id
#TODO: this function should be improved
return node_list[0]
def unstage_node_execution(self):
assert self.staged_node_id is not None
self.staged_node_id = None
def is_staged_node_blocked(self):
assert self.staged_node_id is not None
return self.blockCount[self.staged_node_id] > 0
def complete_node_execution(self):
node_id = self.staged_node_id
self.pop_node(node_id)
self.execution_cache.pop(node_id, None)
self.execution_cache_listeners.pop(node_id, None)
self.staged_node_id = None
def get_nodes_in_cycle(self):
# We'll dissolve the graph in reverse topological order to leave only the nodes in the cycle.
# We're skipping some of the performance optimizations from the original TopologicalSort to keep
# the code simple (and because having a cycle in the first place is a catastrophic error)
blocked_by = { node_id: {} for node_id in self.pendingNodes }
for from_node_id in self.blocking:
for to_node_id in self.blocking[from_node_id]:
if True in self.blocking[from_node_id][to_node_id].values():
blocked_by[to_node_id][from_node_id] = True
to_remove = [node_id for node_id in blocked_by if len(blocked_by[node_id]) == 0]
while len(to_remove) > 0:
for node_id in to_remove:
for to_node_id in blocked_by:
if node_id in blocked_by[to_node_id]:
del blocked_by[to_node_id][node_id]
del blocked_by[node_id]
to_remove = [node_id for node_id in blocked_by if len(blocked_by[node_id]) == 0]
return list(blocked_by.keys())