bad content

[jmottishaw]

Azure Document Intelligence is falling on tables that have blank values in them.

the bad 'content' details are polluting my outputs. I had to implement something like this:

                            table_only_data = {
                                "content": "\n".join([", ".join(row) for row in single_page_data.get("tables", [])]),  # Tables as content
                                "filename": single_page_data.get("filename"),
                                "pageNo": single_page_data.get("pageNo")
                            }

I tried using description fields to tell the LLM to only use table data and not content but that failed miserably. Is there a better way to do this?

[enoch3712]

Hello @jmottishaw !

Can you be more specific? Right now for Azure Document Intelligence, you can only use "prebuilt-layout", that will change in the future.

This will get you the overall content of the page.

Now, can you be more specific? Are you getting the content from the load:

document_loader = DocumentLoaderAzureForm(
    credentials=os.getenv("AZURE_FORM_RECOGNIZER_KEY"),
    endpoint=os.getenv("AZURE_FORM_RECOGNIZER_ENDPOINT")
)

content = document_loader.load_file(test_file_path)

Or are you doing normal extraction? If you have problem a in the structure, i strongly advice to use vision:

result = extractor.extract(test_file_path, InvoiceContract, vision=True)

[jmottishaw]

I am using:

                document_data = azure_loader.load(temp_file_path)
                total_pages = len(document_data)
                page_indices = determine_pages_to_process(total_pages, extract_pages)

                formatted_pages = [
                    {
                        "tables": document_data[i - 1].get("tables", []),
                        "content": document_data[i - 1].get("content", "").strip(),
                        "filename": blob.name,
                        "pageNo": i
                    }
                    for i in page_indices
                ]

The issue is that 'content' is good 95% of the time until it's not.

In my model:

pressure_kpa: float = Field(
description="Extract this value from the 'Pressure (kPa)' column in the tables. Do not use values from content."
)

The tables data is good. Content values are bad. The model always responds with the bad values from content. I am using vision=true.

Using azure/gpt-4o as the LLM

[enoch3712]

Interesting!

The code is simple:

https://github.com/enoch3712/ExtractThinker/blob/main/extract_thinker/document_loader/document_loader_azure_document_intelligence.py

Should be able to be always predictable. Maybe i need to change something, but i built it according to the standards ("maybe not").

 I am using vision=true.

Using azure/gpt-4o as the LLM

If you are doing this the problem should be from the Azure, because the LLM can see from both places.

Can you point to an use case, if is possible/public, where some data is missing as you mention? I would be great.

If the content is not there, while i fix it, i advice you to use just vision, without a documentLoader, or something simple, like pypdf documentLoader