Training data format

[chiefMarlin]

Hi,
Can someone give me some pointers on how the training data gets broken up ?
Example here https://github.com/ggerganov/llama.cpp/tree/master/examples/finetune reads in full blob of text from here
https://raw.githubusercontent.com/brunoklein99/deep-learning-notes/master/shakespeare.txt
But looking at the examples of fine tuning data on huggingface typically training data is supplied in json which is then split by objects that contain input/output.
So in my case i am trying to train mistral 7b model, does anyone know how i would store the data in the training file ?
Would splitting individual examples by newline be sufficient ?

Eg:

<s>[INST] What is the moon made of ? [/INST] Its made from green cheese. <\s>
<s>[INST] What color is the moon ? [/INST] Its the color of green cheese. <\s>
<s>[INST] Why do astronauts like the moon ? [/INST] Because they like to throw cheese parties there. <\s>
<s>[INST] What kind of cheese is the moon made of ? [/INST] Green cheese. <\s>

[m1chae1bx]

I think you can use the --sample-start=“<s>”

train.cpp

       // find all positions of pattern sample_start
        size_t sample_begin = data_str.find(sample_start, 0);
        while (sample_begin != std::string::npos) {
            out_samples_begin.push_back(sample_begin);
            const size_t search_start = sample_begin + sample_start.size();
            sample_begin = data_str.find(sample_start, search_start);
        }
        if (out_samples_begin.size() == 0) {
            printf("%s: warning: sample start pattern '%s' not found. inserting single sample at data begin\n",
                __func__, sample_start.c_str());
            out_samples_begin.push_back(0);
        }

[micheledellaguardia]

Sadly, just adding --sample-start=“<s>” doesn't work, since the newly trained model gives me those
(or part of them) in the replies...
So, cannot understand which is the right format for Q&A. Any idea?