Speed up rebuild and load of index

[nshern]

As referenced here I am persisting my index and re-using it when my app is rerun. It takes about a minute for this process to complete - is there anyway I can speed up this process?

I am using a streamlit frontend and would like to prevent the app from hanging for a minute on start-up.

[dosubot[bot]]

🤖

Hey @nshern! Great to see you diving deeper into LlamaIndex. How's everything going on your end?

To speed up the rebuild and load process of the LlamaIndex data framework and prevent your streamlit frontend application from hanging on start-up, you might consider adjusting the number of workers in the run_jobs function. The number of workers determines how many tasks can be run concurrently. Increasing the number of workers can speed up the process if your system has the resources to handle more concurrent tasks.

Here's how you can do it:

# Increase the number of workers
DEFAULT_NUM_WORKERS = 10  # or any number based on your system's capacity

async def run_jobs(
    jobs: List[Coroutine],
    show_progress: bool = False,
    workers: int = DEFAULT_NUM_WORKERS,
) -> List[Any]:
    """Run jobs.

    Args:
        jobs (List[Coroutine]):
            List of jobs to run.
        show_progress (bool):
            Whether to show progress bar.

    Returns:
        List[Any]:
            List of results.
    """
    asyncio_mod = get_asyncio_module(show_progress=show_progress)
    semaphore = asyncio.Semaphore(workers)

    async def worker(job: Coroutine) -> Any:
        async with semaphore:
            return await job

    pool_jobs = [worker(job) for job in jobs]

    return await asyncio_mod.gather(*pool_jobs)

Please note that increasing the number of workers will increase the usage of system resources. Make sure your system has enough resources to handle the increased number of workers. You can find this in the llama_index/async_utils.py file.

The current default number of workers set in the run_jobs function in the LlamaIndex codebase is 4. This is indicated by the line DEFAULT_NUM_WORKERS = 4 and the function signature async def run_jobs(jobs: List[Coroutine], show_progress: bool = False, workers: int = DEFAULT_NUM_WORKERS,) -> List[Any]: where workers is set to DEFAULT_NUM_WORKERS by default.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• llama_index/async_utils.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.