Distributed batch processing

[dolfim-ibm]

This threads defines the roadmap for batch processing within the Docling package.

In general, the first priority is to optimize the resource usages when converting each document. See the respective issues outlining benchmarks and optimization strategies #306.

Distributed batch processing within Docling

We are currently not planning of adding support for distributed batch processing within Docling. This would add more complexity and increase the base dependencies, which is not what we are looking for.

The advantage of embedding distributed batch processing would be sharing and reusing model instances. The models used by Docling are designed to minimize the required resources, hence we don't expect this would bring any big benefit.

Distributed batch processing around Docling

We encourage the usage of Docling within distributed pipelines which parallelize the documents being processed. We will also do our best to support the requirements of these frameworks, in order to simplify the adoption of Docling.

In particular, we are also planning to provide a few out-of-the-box examples on scheduling via Ray.

Data-prep-kit

Our colleagues are actively working on a simple scalable toolkit for scaling out the data workflows, called data-prep-kit (DPK).

Docling is available as a transform in DPK which allows to launch conversion of large amount of documents on a distributed infrastructure. This is currently being used to process +1 billion documents.

[ashwan1]

I recently used Ray data to parse thousands of documents in parallel using Docling. The documents were, on average, 300 pages long. It took around 1 hr to process 1 document on an Intel x64, 12-core, 16 G RAM machine with no GPUs. However, parallelization using Ray worked fine and saved me a lot of time.
I can contribute an example if that is the plan.

[simjak]

That's great, could you share the example?

[ashwan1]

Sure, will share the repo soon.

[simjak]

hey @ashwan1 would it be possible to have the repo?
I'm choosing between Dask and Ray (I have no experience with any of them :) )

[ctandrewtran]

Plus 1 here! Interested

[archasek]

@ashwan1 we're begging! :D

[simjak]

@dolfim-ibm, is there an example of how I can use the Data Prek kit with Docling using Ray?

[dolfim-ibm]

In Data Prep Kit you find Docling in the pdf2parquet transform. I think this RAG example is using Ray; the first step is Docling https://github.com/IBM/data-prep-kit/blob/dev/examples/notebooks/rag/rag_1A_dpk_process_ray.ipynb.

[pbonito]

@dolfim-ibm anything available for apache-beam/dataflow?