|
| 1 | +<div align="center"> |
| 2 | + |
| 3 | +# nonebot-plugin-fishspeech-tts |
| 4 | + |
| 5 | +<a href="https://v2.nonebot.dev/store"><img src="https://count.getloli.com/get/@nonebot-plugin-fishspeech-tts?theme=asoul"></a> |
| 6 | + |
| 7 | +_⭐ A TTS plugin based on Nonebot2 that calls online [fish-audio](https://fish.audio/zh-CN/) or offline [fish-speech](https://github.com/fishaudio/fish-speech) APIs ⭐_ |
| 8 | +_⭐Text-to-speech `tts` plugin⭐_ |
| 9 | + |
| 10 | +<a href="https://www.python.org/downloads/release/python-390/"><img src="https://img.shields.io/badge/python-3.10+-blue"></a> <a href=""><img src="https://img.shields.io/badge/QQ-1141538825-yellow"></a> <a href="https://github.com/Cvandia/nonebot-plugin-game-torrent/blob/main/LICENCE"><img src="https://img.shields.io/badge/license-MIT-blue"></a> <a href="https://v2.nonebot.dev/"><img src="https://img.shields.io/badge/Nonebot2-2.2.0+-red"></a> |
| 11 | + |
| 12 | +[**简体中文**](README_EN.md) | **English** |
| 13 | +</div> |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +## ⭐ Introduction |
| 18 | + |
| 19 | +**With just a 5-second voice sample, you can ~~perfectly~~ excellently clone the original voice!** |
| 20 | +Simply prepare the voice of the character you wish to clone, label the voice file (see below), and you can quickly generate speech. |
| 21 | + |
| 22 | +> Alternatively, use the official online API -> [fish-audio](https://fish.audio/zh-CN/) for fast cloud-based speech generation. |
| 23 | +
|
| 24 | +## 📜 Disclaimer |
| 25 | + |
| 26 | +> [!note] |
| 27 | +> This plugin is for **learning** and **research** purposes only. Users must assume all risks associated with using the plugin. The author is not responsible for any losses or issues arising from the use of the plugin. Please use the plugin responsibly and **comply with relevant laws and regulations.** |
| 28 | +Using **this plugin indicates that you have read and agree to the above disclaimer**. If you do not agree or cannot comply with the above statement, please do not use this plugin. |
| 29 | + |
| 30 | +## 💿 Installation |
| 31 | + |
| 32 | +<details> |
| 33 | +<summary>installation</summary> |
| 34 | + |
| 35 | +**Using `pipx`:** |
| 36 | +```bash |
| 37 | +pipx install nonebot-plugin-fishspeech-tts -U |
| 38 | +``` |
| 39 | +> [!note] Add this plugin to the `plugins = ["xxx"]` section in `pyproject.toml`. |
| 40 | +
|
| 41 | +**Using `nb-cli`:** |
| 42 | +```bash |
| 43 | +nb plugin install nonebot-plugin-fishspeech-tts -U |
| 44 | +``` |
| 45 | + |
| 46 | +**Using `git clone` (not recommended):** |
| 47 | +- Run the following command in the command prompt: |
| 48 | +```bash |
| 49 | +git clone https://github.com/Cvandia/nonebot-plugin-fishspeech-tts |
| 50 | +``` |
| 51 | +- Copy the `nonebot-plugin-fishspeech-tts` folder to the `src/plugins` directory in the bot's root directory (or another name you used when creating the bot). |
| 52 | + </details> |
| 53 | + |
| 54 | + <details> |
| 55 | + <summary>note</summary> |
| 56 | + |
| 57 | +Recommended mirror sites for download: |
| 58 | + |
| 59 | +- Tsinghua Source: `https://pypi.tuna.tsinghua.edu.cn/simple` |
| 60 | +- Alibaba Source: `https://mirrors.aliyun.com/pypi/simple/` |
| 61 | + |
| 62 | +</details> |
| 63 | + |
| 64 | +## ⚙️ Configuration |
| 65 | + |
| 66 | +**Add the following configurations to `.env`:** |
| 67 | + |
| 68 | +| Basic Configuration | Type | Required | Default Value | Description | |
| 69 | +|:-----:|:----:|:----:|:---:|:----:| |
| 70 | +| tts_is_online | bool | Yes | True | Whether to use the cloud API | |
| 71 | +| tts_chunk_length | literal | No | "normal" | Audio chunk length for requests, defaults to normal; options: short, normal, long | |
| 72 | +| tts_is_to_me | bool | No | True | Whether to respond only when mentioned | |
| 73 | +| tts_audio_path | str | No | "./data/reference_audio" | Path for voice samples, defaults to "./data/reference_audio" | |
| 74 | + |
| 75 | +**Note:** The file name format for reference audio should be: `[Character Name]Text Label.[Audio Extension]` |
| 76 | + |
| 77 | +**! Supports different voices for the same character!** |
| 78 | + |
| 79 | +**Currently supported audio extensions are detailed in [files.py](./nonebot_plugin_fishspeech_tts/files.py) under `AUDIO_FILE_SUFFIX`.** |
| 80 | + |
| 81 | +--- |
| 82 | + |
| 83 | +If you want to use the official API, set the configuration item `tts_is_online` to `True` and configure the following: |
| 84 | + |
| 85 | +| Configuration Item | Type | Required | Default Value | Description | |
| 86 | +|:-----:|:----:|:----:|:---:|:----:| |
| 87 | +| online_authorization | str | Yes | "xxxxx" | API key for official authorization, see [link](https://fish.audio/zh-CN/go-api/api-keys/) | |
| 88 | +| online_model_first | bool | No | True | Set this to `False` if you want to call the official model with your reference audio to customize character voice; if you don't have reference audio, it will call existing voices from the official website. See [link](https://fish.audio/zh-CN/) for details. | |
| 89 | + |
| 90 | +--- |
| 91 | + |
| 92 | +If you want to use a [self-hosted](#offline-setup-fish-speech) or other [fish-speech](https://github.com/fishaudio/fish-speech) project APIs, set the configuration item `tts_is_online` to `False` and configure the following: |
| 93 | + |
| 94 | +| Configuration Item | Type | Required | Default Value | Description | |
| 95 | +|:----:|:----:|:----:|:---:|:----:| |
| 96 | +| offline_api_url | str | Yes | "http://127.0.0.1:8080" | Your `fish-speech` API address | |
| 97 | + |
| 98 | +## ⭐ Usage |
| 99 | + |
| 100 | +> [!note] |
| 101 | +> Please pay attention to your `COMMAND_START` and the above configuration items. |
| 102 | +
|
| 103 | +### Commands: |
| 104 | + |
| 105 | +| Command | Requires @ | Scope | Description | Permissions | |
| 106 | +|:---:|:---:|:---:|:---:|:---:| |
| 107 | +| xxx says xxx | Based on configuration | all | TTS speech generation | all | |
| 108 | +| Voice list | Yes | all | Retrieve all character lists | all | |
| 109 | +| Voice balance | Yes | all | Check API balance | all | |
| 110 | + |
| 111 | +## 🌙 Future |
| 112 | +- No plans at the moment. |
| 113 | +If you like it, remember to give it a star ⭐ |
| 114 | + |
| 115 | +## 💝 Special Thanks |
| 116 | + |
| 117 | +- [x] [nonebot2](https://github.com/nonebot/nonebot2): The foundation of this project, a very useful chatbot framework. |
| 118 | +- [x] [fish-speech](https://github.com/fishaudio/fish-speech): Zero-shot & few-shot TTS: Generate high-quality TTS output with just 10 to 30 seconds of voice samples. |
| 119 | + |
| 120 | +## ⭐ Additional Help |
| 121 | + |
| 122 | +### Brief Tutorial for Offline Setup |
| 123 | + |
| 124 | +**Prepare `fish-speech`:** |
| 125 | +1. Clone the `fish-speech` repository locally: |
| 126 | +```bash |
| 127 | +git clone https://github.com/fishaudio/fish-speech |
| 128 | +``` |
| 129 | +2. Run `install_env.bat` to set up the virtual environment and required dependencies. |
| 130 | +3. Run `start.bat` for the initial startup. |
| 131 | +4. Modify `API_FLAGS.txt` and restart. |
| 132 | + |
| 133 | +**Start API Service:** |
| 134 | +1. Modify `API_FLAGS.txt` to uncomment (remove `#`) in front of `api`: |
| 135 | +``` |
| 136 | +# --infer |
| 137 | +--api |
| 138 | +--listen 0.0.0.0:8080 \ # Listening interface |
| 139 | +... |
| 140 | +``` |
| 141 | + |
| 142 | +**Add Additional Parameters in `API_FLAGS.txt`:** |
| 143 | +1. `--compile` -> Whether to start the compiled model (faster TTS generation but slower startup). |
| 144 | +2. `--workers <number>` -> Start `<number>` of concurrent processes (must set, as default one can easily block). |
| 145 | +3. More details can be found in the [official documentation](https://speech.fish.audio/zh). |
0 commit comments