Something is change in libraries

[mohamedae17]

This part was working last March on colab. Now I can't.. Is there anything that has changed in the way libraries are called?
the code :

!pip install transformers
!pip install protobuf>=3.20.3
!pip install tensorflow_text
!pip install Sentencepiece
!pip install git+https://github.com/deepset-ai/haystack.git
! wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.2-linux-x86_64.tar.gz -q
! tar -xzf elasticsearch-7.9.2-linux-x86_64.tar.gz
! chown -R daemon:daemon elasticsearch-7.9.2
from haystack.utils import clean_wiki_text,convert_files_to_docs,fetch_archive_from_http,print_answers
from haystack.nodes import FARMReader,TransformersReader,BM25Retriever
from haystack.document_stores.elasticsearch import ElasticsearchDocumentStore
from haystack.document_stores import ElasticsearchDocumentStore
import os
from subprocess import Popen,PIPE,STDOUT

the error :

---

ModuleNotFoundError Traceback (most recent call last)
in <cell line: 9>()
7 get_ipython().system(' tar -xzf elasticsearch-7.9.2-linux-x86_64.tar.gz')
8 get_ipython().system(' chown -R daemon:daemon elasticsearch-7.9.2')
----> 9 from haystack.utils import clean_wiki_text,convert_files_to_docs,fetch_archive_from_http,print_answers
10 from haystack.nodes import FARMReader,TransformersReader,BM25Retriever
11 from haystack.document_stores.elasticsearch import ElasticsearchDocumentStore

5 frames
/usr/local/lib/python3.9/dist-packages/haystack/nodes/file_converter/base.py in
3 from abc import abstractmethod
4 from pathlib import Path
----> 5 import langdetect
6 from tqdm.auto import tqdm
7 from haystack.nodes.base import BaseComponent

ModuleNotFoundError: No module named 'langdetect'

---

NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

[anakin87]

Hello @mohamedae17!

Recently the core dependencies of Haystack have been reduced, in order to make the installation faster and simpler.
See

haystack/pyproject.toml

Lines 153 to 155 in dbe3049

	preprocessing = [
	"nltk",
	"langdetect", # for language classification

You can still install langdetect (the missing dependency) via pip install langdetect (or pip install farm-haystack==1.15.1[preprocessing] ).

In any case, to ensure future reproducibility, I suggest pinning your Haystack version instead of installing it from main.
For example:
pip install farm-haystack==1.15.1 instead of pip install git+https://github.com/deepset-ai/haystack.git

[mohamedae17]

Thanks @anakin87
This was useful for some libraries but now the problem is with this library

from haystack.nodes import ElasticsearchRetriever

Error :

ImportError: cannot import name 'ElasticsearchRetriever' from 'haystack.nodes' (/usr/local/lib/python3.9/dist-packages/haystack/nodes/init.py)

[anakin87]

I think the new name of this node is BM25Retriever. See https://docs.haystack.deepset.ai/reference/retriever-api#bm25retriever

[mohamedae17]

It works ! .. thanks alot ♥️

[ZanSara]

Thanks a ton @anakin87 for the help!

[anakin87]

@ZanSara I was thinking: is this behavior correct/intended?

1. install haystack from main (pip install git+https://github.com/deepset-ai/haystack.git)
2. import haystack
3. ModuleNotFoundError: No module named 'langdetect'

[ZanSara]

No it isn't! I must have forgotten to make that import optional. I'm going to make a quick fix for it. Thanks for this!

Crazy that we did not see anything failing in CI... I might also add a CI task to check that the basic Haystack import always works. We're gonna need it more and more.