Liked our work? give us a ⭐!
TThis repository contains an easy-to-use and understand code to instruct-tune, or SFT (supervised fine tuning) TinyLlama to create ChatGPT like chatbots (stateless).
I'll be using TinyLlama, however as stated in their repository it uses the same architecture as Llama 2. If you want to use Llama 2 it'll be probably fine.
We'll be using both Alpaca Dataset, and a custom dataset that we'll load from our local machine.
Prompt format we'll be using is
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Input:
{context}
### Response:
There are lots of prompting formats. I explained detailly when to use which prompting format detailly in the video.
