train-text-from-scratch/finetune Clarification Questions

[braden-dev]

I am relatively new to this LLM world and the end goal I am trying to achieve is to have a LLaMA 2 model trained/fine-tuned on a text document I have so that it can answer questions about it. Ideally, the model would only be able to answer questions about the specific information I give it so it can't answer incorrectly or respond with information that isn't applicable.

The train-text-from-scratch program seemed to do just this; however, since it uses the basic llama vocab file, the model that's output doesn't seem to have an understanding of the English language. I'm assuming since the model wasn't pretrained, it doesn't have the ability to understand text and reply correctly? (I'm not entirely sure about this, just my speculation)

I've also tried using the finetune program to finetune the LLaMA 2 HF model: TheBloke/firefly-llama2-13B-chat-GGUF. After finetuning, .bin and .gguf files were created; however, upon inferencing the model did not seem to know any of the information in the text file I gave it.

I trained it on a 311KB text file containing a guide for an organization I am working for.

Any help to get me moving in the right direction would be greatly appreciated!

Update:
I tried another finetuning run. I trained the TheBloke/Llama-2-13B-chat-GGUF model on a file containing data about a game I made up with a random name to see if the fine-tuned GGUF model would have any knowledge of it and upon inferencing, it made up different answers each time I asked questions about information it was supposed just finetuned with. When starting up the server to inference I tried using the default --lora flag with the weight of 1.0 as well as the --lora-scaled flag with weights of 2 and 5 with the same results each time.

Commands I ran:

• to finetune the model, I ran:
  • ./finetune --model-base ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M-LATEST.gguf --checkpoint-out ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M-ITERATION.gguf --lora-out ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M-ITERATION.bin --train-data "../training-text/flanjit-club.txt" --save-every 0 --threads 6 --adam-iter 30 --batch 4 --ctx 64 --use-checkpointing
• to start the server for inferencing
  • ./server.exe -m ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M.gguf --lora ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M-LATEST.bin --n-gpu-layers 32 -c 2048
  • ./server.exe -m ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M.gguf --lora-scaled ..\Models\llama2_13b\llama-2-13b-chat.Q5_K_M-LATEST.bin 5 --n-gpu-layers 32 -c 2048

[micheledellaguardia]

Did you achieved some (correct) results training from scratch?