mistral-common is a set of tools to help you work with Mistral AI models.
We open-source the tokenizers, validation and normalization code that can be used with our models.
This ensures that you can take full advantage of our models for the following features:
- tokenization of text, images and tools calls.
- validation and normalization of requests, messages, tool calls, and responses. This is built on top of the Pydantic library.
We also version our tokenizers to guarantee backward compatibility for the models that we release.
This library is for you if you want to:
- use our models in your own application.
- build your own models and want to use the same tokenization and validation code as we do.
You can install the library using pip:
pip install mistral-common[opencv]For more information, please refer to the documentation.
We welcome contributions to this library. Please open an issue on our GitHub repository if you have any questions or suggestions.
All of our features are tested to ensure best usage. But if you encounter a bug, find difficulties in using mistral-common. Please also open an issue.
This library is licensed under the Apache 2.0 License. See the LICENSE file for more information.