Mixtral model lineage (mlx_lm.models.mixtral)

[uogbuji]

Notes, considerations, etc. around models derived from Mistral AI's Mixtral.

[uogbuji]

Salesforce's xLAM (Mixtral based) isn't really cooperating with OpenAI style, but I had figured I'll need specialized capability for it, anyway.

❯ python -m mlx_lm.convert --hf-path Salesforce/xLAM-v0.1-r --mlx-path ~/.local/share/models/mlx/xLAM-v0.1-r-4bit -q
…
❯ python test/quick_check.py $HOME/.local/share/models/mlx/xLAM-v0.1-r-4bit/
Model type: mixtral
 Hello! I'm just an AI language model, so I don't have feelings or emotions. But I'm here to help you with any questions or tasks you have! How can I assist you today?
 ======================================== Country extraction
 [{"name": "Nigeria", "continent": "Africa"}]
 ======================================== Square root of 256, pt 1
⚙️ Calling tool square_root with args {'square': 256}
⚙️ Tool call result: 16.0

 ======================================== Square root of 256, pt 2

 ======================================== Usain bolt

 ======================================== END CHECK

[uogbuji]

Salesforce released a new crop of xLAM models. Note that they're non-commercial license. Here's a HF Space which gives a good view of the trained patterns.