AI Generated Exams Reproducibility and Data

A Large Scale Field Study of AI-Generated Exams — Replication Materials

Data and replication materials for the IRT analyses described in "A Large Scale Field Study of AI-Generated Exams".

Instructions

To reproduce the analyses:

1. Install R (version 4.4.0 or later) and make sure you have make installed.

2. Install required R packages:
   From the root directory, run:

   make install_r_packages
   

3. Run the full analysis and generate plots:
   From the root directory, run:

   make clean
   make
   

   This will:

   • Reprocess the raw data
   • Fit the Bayesian 2PL IRT model
   • Regenerate all plots in the plots/ directory

---

Note:
For code and documentation related to the generation of AI exams (not the IRT analysis), see the /exam_generation folder and the README files within each respective subfolder. These scripts are provided for transparency and description only—they will not run as distributed, since we cannot share the original syllabus or exam materials from our field study participants.