Docling deployment CPU only in AWS Lambda (python 3.12)

[sanhuezapablo]

Has anyone tried deploying Docling as an AWS Lambda via docker? Would appreciate the help with this. I'm having issues with easyOCR, see error below. I can provide more details if needed, just having a hard time with setting model paths for docling and easyOCR.

{
  "errorMessage": "[Errno 30] Read-only file system: '/home/sbx_user1051'",
  "errorType": "OSError",
  "requestId": "ac4853c4-7c02-4ad1-b2aa-0ba668e94962",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 140, in lambda_handler\n    processing_time = process_pdf(input_path, output_dir)\n",
    "  File \"/var/task/lambda_function.py\", line 72, in process_pdf\n    conv_res = converter.convert(input_path)\n",
    "  File \"/var/lang/lib/python3.12/site-packages/pydantic/_internal/_validate_call.py\", line 38, in wrapper_function\n    return wrapper(*args, **kwargs)\n",
    "  File \"/var/lang/lib/python3.12/site-packages/pydantic/_internal/_validate_call.py\", line 111, in __call__\n    res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 212, in convert\n    return next(all_res)\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 235, in convert_all\n    for conv_res in conv_res_iter:\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 270, in _convert\n    for item in map(\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 311, in _process_document\n    conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 332, in _execute_pipeline\n    pipeline = self._get_pipeline(in_doc.format)\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/document_converter.py\", line 298, in _get_pipeline\n    self.initialized_pipelines[pipeline_class] = pipeline_class(\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/pipeline/standard_pdf_pipeline.py\", line 81, in __init__\n    if (ocr_model := self.get_ocr_model(artifacts_path=artifacts_path)) is None:\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/pipeline/standard_pdf_pipeline.py\", line 170, in get_ocr_model\n    return EasyOcrModel(\n",
    "  File \"/var/lang/lib/python3.12/site-packages/docling/models/easyocr_model.py\", line 76, in __init__\n    self.reader = easyocr.Reader(\n",
    "  File \"/var/lang/lib/python3.12/site-packages/easyocr/easyocr.py\", line 65, in __init__\n    Path(self.user_network_directory).mkdir(parents=True, exist_ok=True)\n",
    "  File \"/var/lang/lib/python3.12/pathlib.py\", line 1315, in mkdir\n    self.parent.mkdir(parents=True, exist_ok=True)\n",
    "  File \"/var/lang/lib/python3.12/pathlib.py\", line 1315, in mkdir\n    self.parent.mkdir(parents=True, exist_ok=True)\n",
    "  File \"/var/lang/lib/python3.12/pathlib.py\", line 1311, in mkdir\n    os.mkdir(self, mode)\n"
  ]
}

[sanhuezapablo]

I got this to work with tessaract and easyOCR via docker. Taking a look at docling-serve helped out a lot here if anyone is wondering.

[MatheusAbdias]

@sanhuezapablo And about you error using AWS Lambda, its a permission this error occurs because AWS Lambda runs in an environment where the base file system is read-only. Your code is trying to create directories or files in /home/sbx_user1051, which is a protected area of the system.

[ColeDrain]

@sanhuezapablo If you want to do OCR on your documents with great speed, you really need to use Accelerator options with CUDA. You can check the docs for details. In my own experience, using CUDA with EasyOCR processes an average of 100 pages per minute. Other important point is about using docker, you need to setup nvidia container tool kit for allow containers to using GPU you can check in this docs.

Hi @MatheusAbdias. 100 pages per minute is nice, can you share the config you used, I am struggling with getting decent processing time.
And I would appreciate if you could share your Dockerfile setup..

[sanhuezapablo]

Thanks @MatheusAbdias! I figured that out fairly quickly (about the error). I still had other issues but I resolved those already.
Yeah, I do know that using CUDA would make processing much more efficient. I was mostly testing and playing around just to get something going. I'm thinking of deploying something with CUDA in AWS (maybe a modified version of docling-serve to fit our use cases). What type of GPU would you recommend? have you done any load testing? any other hints/suggestions?

[jjanczur]

How did you deploy it in a lambda? I'm hitting the 250 MB size limit.

[nithingovindugari]

@sanhuezapablo Thanks for sharing your experience with deploying docling on Lambda via container images. I'm working on a similar implementation with a large container image (~3GB) and I'm concerned about cold start performance.
I noticed you mentioned an 8-10 second startup time for new Lambda instances.

1. Want to know if you noticed if Lambda actually downloads the entire container image for each cold start, or does it seem to use some kind of cached version after the initial deployment?

2. Are you still proceeding with Lambda ( Any other AWS service ) and do you see any improvement in speed?

3. How did you adapt docling-serve for your use case?

Thanks for your time.

[nikhilkudavemnh]

this will solve your problem, you must be using ecr image for lambda

os.environ["HF_HOME"] = "/tmp/huggingface"
os.environ["EASYOCR_MODULE_PATH"] = "/tmp"
os.environ["MODULE_PATH"] = "/tmp"

[DanielTheisen]

Hi,
could you give some more details what you have done?
Did you just added this three env-variables ?
Have you specified anything in the pipeline_options? Like model_storage_directory

When I just add these three lines I still get the same error. I added this in the main.py which includes the code and build a docker image with the code and dependencies and upload them to ECR.