Aysnc reader node

[nasrin-taghizadeh]

In my pipeline, I defined a reader node which sends all requests to a remote server, waits for answers and return them upon receiving them. so I need to define predict function of reader class async. However, defining async predict function rises error:

File "/usr/local/lib/python3.7/site-packages/haystack/nodes/reader/base.py", line 96, in run
BaseReader.add_doc_meta_data_to_answer(documents=documents, answer=answer) for answer in results["answers"]
TypeError: 'coroutine' object is not subscriptable

any idea is appreciated.

[tstadel]

@nasrin-taghizadeh the predict function is not intended to be async. The whole pipeline can only proceed when the prediction was made. So we wouldn't gain too much by making it async-compatible. You can make a blocking call to the remote server however.
If you want to make multiple predictions at once, check out the predict_batch function. Also this function is not async. But you could fire multiple requests from it, collect them and return the collected batch results.
If you're interested in the latter approach, checkout how asyncio.run is being used in https://blog.devgenius.io/how-to-send-concurrent-http-requests-in-python-d9cda284c86a

[nasrin-taghizadeh]

Thank you for your consideration.