fix(processor): race condition that could result in node errors not getting reported

I had set the cancel event at some point during troubleshooting an unrelated issue. It seemed logical that it should be set there, and didn't seem to break anything. However, this is not correct.

The cancel event should not be set in response to a queue status change event. Doing so can cause a race condition when nodes are executed very quickly.

It's possible that a previously-executed session's queue item status change event is handled after the next session starts executing. The cancel event is set and the session runner sees it aborting the session run early.

In hindsight, it doesn't make sense to set the cancel event here either. It should be set in response to user action, e.g. the user cancelled the session or cleared the queue (which implicitly cancels the current session). These events actually trigger the queue item status changed event, so if we set the cancel event here, we'd be setting it twice per cancellation.

Author: psychedelicious

invokeai/app/services/session_processor/session_processor_default.py

@@ -374,7 +374,6 @@ async def _on_queue_event(self, event: FastAPIEvent) -> None:
             "failed",
             "canceled",
         ]:
-            self._cancel_event.set()
             self._poll_now()
 
     def resume(self) -> SessionProcessorStatus: