Official repository of the PhD course "Robust and Reproducible Research” (a.k.a. R^3).
The number of scientific articles published in Computer Science (and similar fields) increases steadily every year. This is mainly due to breakthroughs like Deep Learning, and, more recently, Large Language Models.
Paradoxically, researchers are struggling even more to reproduce published research. This issue affects all possible aspects of research, including methodology, data curation, approach comparison, and implementation.
In this course, we'll introduce and discuss the concept of 'reproducibility' in research. In particular, we'll overview current issues in research and existing attempts to address them. We'll focus on data curation, experimental setup, model comparison, and programming best practices.
This course is recommended for all types of researchers, from those who have just embarked on their journey to those who have always wondered how certain research managed to get published. See Section Prerequisites for more details.
2024-2025--> "Robust and Reproducible Research" (16 hours)2022-2023--> "Robust and Reproducible Experimental Deep Learning Setting" (10 hours)
Down below, you can find the program of the course.
- Lecture 1: Reproducibility in Research (Pt. I)
- Lecture 2: Reproducibility in Research (Pt. II)
- Lecture 3: Data Collection and Annotation
- Lecture 4: Modeling and Experimenting
- Lecture 5: Responsible Research
- Lecture 6: Programming Best Practices (Pt. I)
- Lecture 7: Programming Best Practices (Pt. II)
- Lecture 8: Cinnamon: a lightweight python library for research
Duration: 16 Hours Lecture Format: 2 hour-long hybrid lectures.
Lectures are meant to be interactive.
- Programming: Intermediate
- Deep Learning Theory: Intermediate
- Jupyter Notebook: Beginner
If you need to certify your attendance to the course via an exam, you can submit a review as described below. Please submit your report via email.
- [Review] You can submit a review you have made concerning a paper of your choice. Please submit your review in .pdf format (1-2 pages at most). The review should focus on issues related to the course topics and corresponding solution(s) suggested by you or proposed in the paper.
There is not deadline to submit the report! It is entirely up to you.
Please use the LaTeX template located in 2024-2025/Review Template.
Lecture recordings are publicly available at the following links:
Feel free to contact me via email for any issue/question about the course.