How to know the lenght/size of an index

[aiwalter]

I would like to know the number of chunks/nodes that are in my index. Could not find a simple way to find this out

[dosubot[bot]]

🤖

Hey @aiwalter, great to see you venturing further into the depths of LlamaIndex! 🦙 I'm diving into your query right now and will circle back with something helpful in a jiffy.

[aiwalter]

especially I have issues with e.g. this:

        index.as_query_engine(
            chunk_size=100,
            similarity_top_k=5,
            ResponseMode=ResponseMode.TREE_SUMMARIZE,
        )

as it raises a KeyError: '-1' as the index has less than 5 chunks. Or how can I avoid this issue otherwise? I thought it might be easy fix in case I know the number of chunks in the index to not try fetching more than that. @dosubot

[dosubot[bot]]

🤖

You can determine the size or length of an index in the LlamaIndex codebase by using the all_nodes property of the IndexGraph class. This property contains all the nodes in the index, and its length can be used to determine the size of the index. Here is how you can get the size of the index:

index_graph = IndexGraph()
# ... insert nodes into the index_graph ...
index_size = len(index_graph.all_nodes)

In your case, you can use this to avoid the KeyError: '-1' by checking if the index size is less than the similarity_top_k value. If it is, you can adjust the similarity_top_k value to be equal to the index size.

However, this solution requires you to have access to the IndexGraph object. If you only have the Index object, I couldn't find a direct way to get the size of the index from the Index object in the LlamaIndex codebase.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

You can find more information about the IndexGraph class and its all_nodes property in the llama_index/indices/common_tree/base.py file.

Sources

• llama_index/indices/common_tree/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.