Skip to content

[Feature Request] Add support for batched result retrieval executions #1673

Open
@akaNightmare

Description

@akaNightmare

Is your feature request related to a problem? Please describe.

Hello,
for some reason need to create the report (CSV) from the ALL workflow execution results, how it looks now:

async function* fetchWorkflowExecutionResults({ workflowClient, namespace, companyId }) {
  let nextPageToken: Uint8Array | undefined;
  do {
        const workflows = await workflowClient.workflowService.listWorkflowExecutions({
            namespace,
            query: `ExecutionStatus="Completed" AND WorkflowType="createCompanyWorkflow" AND companyId="${companyId}`,
            nextPageToken,
        });
        nextPageToken = workflows.nextPageToken;

        for (const wf of workflows.executions) {
          const result = await workflowClient
                    .getHandle(wf.execution.workflowId, wf.execution.runId)
                    .result();
          yield {
            company_id: result.company_id,
            field1: result.field1,
            field2: result.field2,
            another_field: result.another_field,
          };
        }
  } while (nextPageToken?.byteLength);
}

it's working as expected for any amount of the executions, but it takes around ~1min to process 1000 executions, so for 1KK executions -> ~1000 minutes, which is not acceptable for my purposes.

Describe the solution you'd like

For faster processing, I can chunk workflows.executions (e.g., in batches of 50 or 100) using Promise.all(). However, I'm not sure if this is the correct behavior — it might make more sense to add a separate method for this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions