How to add a key value pair in metadata json

[deepak-hl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

if file_path.lower().endswith(".pdf"):
            loader = PDFPlumberLoader(file_path)
            document = loader.load()
            print(document,"^^^^^^^^^^^^^^^^^^^^^^^^^^")
            # text_splitter = TokenTextSplitter(chunk_size=2000, chunk_overlap=200)
            text_splitter = RecursiveCharacterTextSplitter(chunk_size=6000, chunk_overlap=100)
            texts = text_splitter.split_documents(documents=document)

Description

How to add a custom key value pair in existing metadata json when running loader.load()

System Info

langchain==0.1.1
langchain-anthropic==0.1.22
langchain-community==0.0.20
langchain-core==0.2.28
langchain-openai==0.0.5
langchainhub==0.1.14
Linux
python==3.12.4

[dosubot[bot]]

To add a custom key-value pair in the existing metadata JSON when running loader.load(), you can use the metadata_func parameter of the PDFPlumberLoader class. Here is an example of how to do it:

from langchain_community.document_loaders import PDFPlumberLoader
from langchain_core.documents import Document

# Define the metadata extraction function.
def metadata_func(record: Document, metadata: dict) -> dict:
    metadata["custom_key"] = "custom_value"  # Add your custom key-value pair here
    return metadata

# Initialize the loader with the metadata function.
loader = PDFPlumberLoader(file_path, metadata_func=metadata_func)
documents = loader.load()

print(documents[0].metadata)

In this example, the metadata_func is used to add a custom key-value pair to the metadata of each document. The metadata_func takes the record and the existing metadata as input and returns the updated metadata. This approach allows you to customize the metadata for each document as needed ^[1][2].

To continue talking to Dosu, mention @dosu.

[deepak-hl]

@dosu, getting below error
PDFPlumberLoader.init() got an unexpected keyword argument 'metadata_func'

[dosubot[bot]]

The PDFPlumberLoader class does not accept a metadata_func argument. To add a custom key-value pair to the metadata, you need to modify the metadata after loading the documents. Here is an example of how to do it:

from langchain_community.document_loaders.pdf import PDFPlumberLoader
from langchain_core.documents import Document

# Initialize the loader without the metadata function.
loader = PDFPlumberLoader(file_path="example.pdf")
documents = loader.load()

# Add custom key-value pair to the metadata of each document.
for document in documents:
    document.metadata["custom_key"] = "custom_value"

print(documents[0].metadata)

This approach allows you to add custom metadata after the documents have been loaded ^[1].

To continue talking to Dosu, mention @dosu.

[deepak-hl]

@dosu getting below error
'tuple' object has no attribute 'page_content'

[dosubot[bot]]

To resolve the error "'tuple' object has no attribute 'page_content'", ensure you are correctly accessing the Document objects. The error suggests you might be trying to access an attribute on a tuple instead of a Document object.

Here's an example of how to correctly add a custom key-value pair to the metadata of documents loaded with PDFPlumberLoader:

from langchain_community.document_loaders.pdf import PDFPlumberLoader

# Initialize the loader
loader = PDFPlumberLoader(file_path="example.pdf")

# Load the documents
documents = loader.load()

# Add custom key-value pair to metadata
for document in documents:
    document.metadata["custom_key"] = "custom_value"

# Now `documents` contains the parsed documents with the custom metadata
for doc in documents:
    print(doc.metadata)

This ensures you are working with Document objects and not tuples ^[1][2].

To continue talking to Dosu, mention @dosu.