Getting error while using confluence loader

[deepak-hl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

confluence_url = config.get("confluence_url", None)
        username = config.get("username", None)
        api_key = config.get("api_key", None)
        space_key = config.get("space_key", None)
        documents = []

        embedding = OpenAIEmbeddings()
        loader = ConfluenceLoader(
            url=confluence_url, username=username, api_key=api_key
        )
        for space_key in space_key:
                    documents.extend(
                        loader.load(
                            space_key=space_key, limit=100
                        )
                    )
                    text_splitter = CharacterTextSplitter(
                        chunk_size=6000, chunk_overlap=50
                    )
                    texts = text_splitter.split_documents(documents)
                    text_splitter = TokenTextSplitter(
                        chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base"
                    )
                    texts = text_splitter.split_documents(texts)

                PGVector.from_texts(
                    embedding=embedding,
                    texts=texts,
                    collection_name=persist_directory,
                    connection_string=CONNECTION_STRING,
                )

Description

getting following error:
Received runtime arguments {'space_key': 'CLI0', 'limit': 100}. Passing runtime args to load is deprecated. Please pass arguments during initialization instead.

System Info

langchain
ubuntu

[dosubot[bot]]

To resolve the deprecation warning and correctly pass the arguments during initialization when using the ConfluenceLoader, you should pass all necessary parameters directly to the ConfluenceLoader constructor instead of passing them to the load method. Here is how you can modify your code:

confluence_url = config.get("confluence_url", None)
username = config.get("username", None)
api_key = config.get("api_key", None)
space_keys = config.get("space_key", None)  # Assuming this is a list of space keys
documents = []

embedding = OpenAIEmbeddings()

# Initialize the ConfluenceLoader with all required parameters
loader = ConfluenceLoader(
    url=confluence_url,
    username=username,
    api_key=api_key,
    limit=100  # Set the limit here
)

for space_key in space_keys:
    loader.space_key = space_key  # Set the space_key for each iteration
    documents.extend(loader.load())

text_splitter = CharacterTextSplitter(
    chunk_size=6000, chunk_overlap=50
)
texts = text_splitter.split_documents(documents)
text_splitter = TokenTextSplitter(
    chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base"
)
texts = text_splitter.split_documents(texts)

PGVector.from_texts(
    embedding=embedding,
    texts=texts,
    collection_name=persist_directory,
    connection_string=CONNECTION_STRING,
)

By passing the limit parameter during the initialization of ConfluenceLoader and setting the space_key attribute for each iteration, you avoid the deprecation warning related to passing runtime arguments to the load method ^[1].

To continue talking to Dosu, mention @dosu.

[deepak-hl]

Received runtime arguments {'include_attachments': True}. Passing runtime args to load is deprecated. Please pass arguments during initialization instead.

[dosubot[bot]]

To avoid the deprecation warning, you should pass the include_attachments parameter during the initialization of ConfluenceLoader instead of passing it to the load method. Here is how you can modify your code:

confluence_url = config.get("confluence_url", None)
username = config.get("username", None)
api_key = config.get("api_key", None)
space_keys = config.get("space_key", None)  # Assuming this is a list of space keys
documents = []

embedding = OpenAIEmbeddings()

# Initialize the ConfluenceLoader with all required parameters
loader = ConfluenceLoader(
    url=confluence_url,
    username=username,
    api_key=api_key,
    limit=100,  # Set the limit here
    include_attachments=True  # Set include_attachments here
)

for space_key in space_keys:
    loader.space_key = space_key  # Set the space_key for each iteration
    documents.extend(loader.load())

text_splitter = CharacterTextSplitter(
    chunk_size=6000, chunk_overlap=50
)
texts = text_splitter.split_documents(documents)
text_splitter = TokenTextSplitter(
    chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base"
)
texts = text_splitter.split_documents(texts)

PGVector.from_texts(
    embedding=embedding,
    texts=texts,
    collection_name=persist_directory,
    connection_string=CONNECTION_STRING,
)

By passing the include_attachments parameter during the initialization of ConfluenceLoader, you avoid the deprecation warning related to passing runtime arguments to the load method ^[1][2].

To continue talking to Dosu, mention @dosu.

[deepak-hl]

getting below error while running the code with include_attachments:
'message'
400 Client Error: Bad Request for url: https://api.media.atlassian.com/file/UNKNOWN_MEDIA_ID/binary?token=eyJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJmMTc3YzU0Ny03Y2NlLTQyN2ItYmQ5Yy00M2I4ZmZlOWRkZDUiLCJhY2Nlc3MiOnsidXJuOmZpbGVzdG9yZTpmaWxlOlVOS05PV05fTUVESUFfSUQiOlsicmVhZCJdfSwiZXhwIjoxNzIzNzE4NzAyLCJuYmYiOjE3MjM2MzU3ODJ9.ZUaXdaEuTxWnqKcIk4rXvSPhBAhmixzWQOPxXzqBbBo&client=f177c547-7cce-427b-bd9c-43b8ffe9ddd5&name=Invalid%20file%20id%20-%20UNKNOWN_MEDIA_ID&max-age=2940