unable to convert a model myself

[maddes8cht]

I am currently trying to convert a Falcon model myself.
I am starting with the original falcon-7b-instruct and falcon-40b-instruct to be sure that problems with the conversion are not caused by possible incompatibilities with a derived model.
I have successfully downloaded the model directory with git clone https://huggingface.co/tiiuae/falcon-40b-instruct and git clone https://huggingface.co/tiiuae/falcon-7b-instruct respectively and successfully performed the conversion with python falcon_convert.py E:/Conversion/falcon-raw/falcon-40b-instruct E:/Conversion/falcon-out use-f32 on my windows system.

But the quantization fails with this message:

falcon_quantize.exe ggml-model-falcon-40b-instruct-f32.bin 2 5
main: build = 881 (66aa59e)
main: quantizing 'ggml-model-falcon-40b-instruct-f32.bin' to 'ggml-model-Q4_0.bin' as Q4_0 using 5 threads
falcon.cpp: loading model from ggml-model-falcon-40b-instruct-f32.bin
falcon.cpp: file version 1 - loading foundation model for quantization only
falcon.cpp: fallback for old file format. Loading BPE merges from tokenizer.json
Error: could not open file ggml-model-falcon-40b-instruct-f32.bin/tokenizer.json
falcon.cpp: error: old file format. Place json data in directory: ggml-model-falcon-40b-instruct-f32.bin/tokenizer.json

It also does not work if I copy the tokenizer.json into the directory of the f32.bin file. Apparently the message says this file should be in a subdirectory with the same name as the f32.bin - but windows won't allow me a directory with the same name as a file in the directory.
Why do I even get the old file format message when I use a freshly converted file?

[cmp-nct]

Hi,
The reason you get the "old file format" warning is because quantize uses the same libfalcon function as main()
It would need a check or define to disable the warning when quantization binary is used (though I want to move the quantization binary into the main soon)
I considered it of low importance, it's the last time the warning is seen.

Regarding the directory: It expects each model to be loaded from a directory
So like falcon_quantize directory/file.bin 2 5
You executed it from within the directory, the logic is simple. It tries to get the path from the model string and looks in the path for tokenizer.json