Hi, for Chinese text processing, can we still only use regular expressions? I wonder if there is a new way now.

[yingxingdechibang]

I'm using haystack to process Chinese and I have the following questions:

1. in retriever, the default NLTK's sent_tokenizer does not support Chinese, so how should we process Chinese, only use regular expressions?
2. in reader, I want to use "Prompts and LLMs", but the documentation says haystack only support "google/flan t_5 models and OpenAI InstructGPT models." I wonder whether it can support other LLMs, like bloom or bloomz？

[bilgeyucel]

Hey @yingxingdechibang,

1. As you said, there is no support for Chinese in PreProcessor. You can use regular expressions or try to split_by = passage in the PreProcessor.
2. Currently, we don't have support for Bloom but we have an issue for it Support for Bloom models #4322 . It will be possible soon :)