Skip to content

[Feature Request] Add support for batched result retrieval executions #1673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akaNightmare opened this issue Apr 10, 2025 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@akaNightmare
Copy link

Is your feature request related to a problem? Please describe.

Hello,
for some reason need to create the report (CSV) from the ALL workflow execution results, how it looks now:

async function* fetchWorkflowExecutionResults({ workflowClient, namespace, companyId }) {
  let nextPageToken: Uint8Array | undefined;
  do {
        const workflows = await workflowClient.workflowService.listWorkflowExecutions({
            namespace,
            query: `ExecutionStatus="Completed" AND WorkflowType="createCompanyWorkflow" AND companyId="${companyId}`,
            nextPageToken,
        });
        nextPageToken = workflows.nextPageToken;

        for (const wf of workflows.executions) {
          const result = await workflowClient
                    .getHandle(wf.execution.workflowId, wf.execution.runId)
                    .result();
          yield {
            company_id: result.company_id,
            field1: result.field1,
            field2: result.field2,
            another_field: result.another_field,
          };
        }
  } while (nextPageToken?.byteLength);
}

it's working as expected for any amount of the executions, but it takes around ~1min to process 1000 executions, so for 1KK executions -> ~1000 minutes, which is not acceptable for my purposes.

Describe the solution you'd like

For faster processing, I can chunk workflows.executions (e.g., in batches of 50 or 100) using Promise.all(). However, I'm not sure if this is the correct behavior — it might make more sense to add a separate method for this.

@akaNightmare akaNightmare added the enhancement New feature or request label Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant