Support for Multithreaded Execution

[luisggc]

Description:

Currently, schemachange executes migration scripts sequentially and as a single process, which can become time-consuming when applying complex schema operations or working with large datasets.

Use case / Motivation:

• In large enterprise environments, schema deployments may involve overhead when applying numerous migrations across large tables.
• Supporting multithreading or batched parallel execution in schemachange could greatly reduce execution time for large-scale deployments, especially when multiple large tables or changes are involved.

Proposed solution:

• Add a configuration option or CLI flag (e.g., --threads N) to enable parallel execution:

Benefits:

• Reduced runtime for long-running migrations.
• Better utilization of CPU and database concurrency.
• Scalable for big migrations.

Potential risks & considerations:

• Increased concurrency stress on Snowflake warehouses. But we can have --threads None by default (current behavior).
• User experience/configuration complexity.

I can work in this feature in case you don't have plans for It.

[rgriffogoes]

Unless things changed, this has been unfortunately decided as "wont fix" - see PR and discussion in #199

[luisggc]

Got It! I am moving hundreds of R__ files to schemachange and in addition to #339 , that could be a time saver.

[MACKAT05]

What about a multi executor?
build a tool that runs schemachange in multiple threads with different folders being the project root targets?