About : A study comparing template-based prompting (TB) with zero-shot (ZS) and few-shot (FS) prompting methods for generating lecture material in computer science education. Developed a React.js frontend to facilitate query submission and result visualization. To check out the frontend , go to the master branch.
Here's the recommended order in which the files should be accessed, along with a brief description of what happens in each file:
1)Dataset_Generation_For_Demonstration.ipynb Description: This notebook focuses on generating datasets using template based prompting required for the analysis. It prepares the data needed for subsequent evaluations.It's not present in the github repo. Purpose: To create datasets for demonstration purposes.
2)TestDataset.ipynb Description: This notebook is used to extract and preprocess test datasets. It ensures that the datasets are properly formatted and ready for evaluation. Purpose: Preprocessing and extraction of test datasets.
3)Rule_Based_Evaluation.ipynb Description: This notebook contains the evaluation of the rule-based approach used in the analysis. It assesses the performance of the rule-based method against predefined criteria. This file is yet to be uploaded. Purpose: Evaluation of the rule-based approach.
4)Gemini_Pro_with_ZS__Ranking_Methodology.ipynb Description: This notebook explores the Gemini Pro with Zero Shot methodology, focusing specifically on ranking. It implements the Zero Shot methodology and ranks the results accordingly. Purpose: Implementation and evaluation of the Gemini Pro with Zero Shot methodology.
5)Comparative_analysis_of_gemini_pro_with_zero_shot.ipynb Description: This notebook contains the main comparative analysis of Gemini Pro with Zero Shot methodology. It compares the results obtained from different approaches and provides insights into their performance. Purpose: Comparative analysis of Gemini Pro with Zero Shot methodology.
6)content__eval__RAG.ipynb Description: This notebook focuses on evaluating the content using the RAG (Red, Amber, Green) rating system. It assesses the quality and relevance of the content generated by the models.This file is yet to be uploaded Purpose: Evaluation of content using the RAG rating system.
7)Comparative_analysis_of_gemini_pro_with_rag.ipynb Description: This notebook conducts a comparative analysis of Gemini Pro with the RAG rating system. It compares the performance of Gemini Pro with other approaches based on the RAG ratings. This file is yet to be uploaded. Purpose: Comparative analysis of Gemini Pro with the RAG rating system.
By following this sequence, users can progressively explore the dataset generation process, model implementations, evaluations, and comparative analyses, gaining insights into the performance of different methodologies and approaches.