[Bug] Upgrading OTel Interceptor to 1.11.5+ breaks signal-with-start and may cause NDE

[mjameswh]

Describe the bug

Assuming a workflow is launched with signalWithStart, the following workflow code in TS SDK 1.11.3 or earlier would result in sequence A-B-C. Same in later releases if not using the OTel workflow interceptor. However, in 1.11.5+ with OTel, we’d get A-C-B (i.e. equivalent of signal coming in a later WFT).

export async function myWorkflow(name: string): Promise<void> {
  log('A');
  setHandler(defineSignal('mySignal'), async () => {
    log('B');
  });
  log('C');
}

This change is due to addition of tracing propagation on inbound signals in 1.11.5 (#1449). Order of promise completion on outbound scheduleLocalActivity has similarly been modified in 1.11.6 (#1577). Both of these changes may result in Non-Determinism errors when older workflows are replayed on newer release of the SDK with the OTel interceptor installed.

For context, the changes mentioned previously added a certain number of await statements to the corresponding code paths, so the actual code is now executed as a microtask rather than synchronously, hence the change in execution order. As a general rule, interceptors provided by the SDK itself should as much as possible avoid adding yield points. We should also document this risk on the Workflow Interceptor types, as user-implemented interceptors may face similar issues.