LLMs as Educational Analysts: Transforming Multimodal Data into Actionable Reading Assessment Reports

Authors

• Eduardo Davalos (ORCID: 0000-0001-7190-7273)
• Yike Zhang (ORCID: 0000-0003-3503-2996)
• Namrata Srivastava (ORCID: 0000-0003-4194-318X)
• Jorge Alberto Salas (ORCID: 0000-0002-8885-0813)
• Sara McFadden (ORCID: 0000-0001-6375-1272)
• Sun-Joo Cho (ORCID: 0000-0002-2600-8305)
• Gautam Biswas (ORCID: 0000-0002-2752-3878)
• Amanda Goodwin (ORCID: 0000-0002-6439-7399)

Affiliation: Vanderbilt University, Nashville, TN 37235, USA

Overview

This repository accompanies the research paper "LLMs as Educational Analysts: Transforming Multimodal Data Traces into Actionable Reading Assessment Reports." The study explores how Large Language Models (LLMs) can synthesize multimodal reading assessment data—such as eye-tracking metrics, learning outcomes, and teaching standards—into teacher-friendly reports to aid classroom instruction.

The project combines unsupervised clustering with LLM-driven report generation, enabling data-driven insights that assist educators in identifying students' reading behaviors and challenges.

Features

• Multimodal Learning Analytics: Uses eye-tracking, reading performance metrics, and assessment standards to analyze student reading behaviors.
• Unsupervised Clustering: Identifies distinct reading behavior profiles using K-Means and Gaussian Mixture Models (GMM).
• LLM-Generated Reports: Converts raw analytics into structured, interpretable insights for educators.
• Teacher Evaluation Pipeline: Includes teacher feedback and LLM-based assessments to refine reports.

Research Findings

• Clustering Effectiveness: K-Means produced the best reading behavior segmentation, with 4 distinct student profiles.
• LLM-Generated Reports: Teachers found clustering insights and content analysis highly useful, but recommended improved readability.
• Human-AI Collaboration: Combining LLM + teacher evaluations refined the clarity and pedagogical relevance of the reports.

Acknowledgments

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A150199 and R305A210347 to Vanderbilt University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.