Transformers and Large Language Models - Winter School - PUCP

Diapositivas para el curso:

• Parte I: Introducción a los Transformers
• Parte II: Modelos Preentrenados
• Parte III: Large Language Models

Código para el curso:

• Parte I: Using Fine-Tuned Models
• Parte II: Fine-Tuning a Model
• Parte III: Further Pre-Training a Model
• Parte IV: Using LLMs
• Parte V: LLMs as Annotators

Enlaces a papers seleccionados:

• Timoneda, Joan C., and Sebastián Vallejo Vera. “BERT, RoBERTa or DeBERTa? Comparing Performance Across Transformer Models in Political Science Text.” The Journal of Politics (2025): 87.1 000-000. PDF
• Dávila Gordillo, Diana, Joan C. Timoneda, and Sebastián Vallejo Vera. “Machines Do See Color: How Transformers Outperform Other Approaches in Classifying Overt and Covert Racism in Text.” (2024). arXiv preprint arXiv:2401.09333. PDF
• Timoneda, Joan C., and Sebastián Vallejo Vera. “Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Tasks.” arXiv preprint arXiv:2503.04874 (2025). PDF
• Vallejo Vera, Sebastián and Hunter Driggers. “Bias in LLMs as Annotators: The Effect of Party Cues on Labelling Decision by Large Language Models.” (2025). arXiv preprint arXiv:2408.15895. PDF