training a new model from nothing

[infdivzero]

The train-text-from-scratch program looks like it should do what I'm looking for but it needs one of the vocab models under models/. I remember reading somewhere in this repo about models typically being developed in something like pytorch then being converted to gguf, but is it possible to develop them using ggml or llama.cpp without building off of one of the vocab models, or even create a new vocab model?

[KerfuffleV2]

The convert.py script has a --vocab-only option, so you can convert for example a HF model to GGUF and only include the metadata. Pretty sure that's also how those vocab only models were created.

So basically two options, find a model that you want to clone the vocab/metadata from and just use that with --vocab-only or just build the vocab/metadata from scratch if you want in a format convert.py can handle. (Or of course write your own script for a completely new format if you want.)

[StuartIanNaylor]

https://github.com/jmorganca/ollama